Differential tracking with a kernel-based region covariance descriptor

Abstract

The covariance descriptor has received an increasing amount of interest in visual tracking. However, the conventional covariance tracking algorithms fail to estimate both the scale and orientation of an object. In this paper, we present a kernel-based region covariance descriptor to address this issue. An affine kernel function is incorporated to the covariance matrix to effectively control the correlations among extracted features inside the object region. Under the Log-Euclidean Riemannian metric, we construct a region similarity measure function that describes the relationship between the candidate and a given appearance template. The tracking task is then implemented by minimizing the similarity measure, in which the gradient descent method is utilized to iteratively optimize affine transformation parameters. In addition, the template is dynamically updated by computing the geometric mean of covariance matrices in Riemannian manifold for adapting to the appearance changes of the object over time. Experimental results compared with several relevant tracking methods demonstrate the good performance of the proposed algorithm under challenging conditions.

Appendix

In this appendix, we describe the first derivative of ${\varvec{C}}_{R} \big (\psi (\varvec{A})\big )$ with respect to $\varvec{A}$. Since

$$\begin{aligned}&\left( \sum \limits _{i=1}^{N} K(\psi (\varvec{A}))(\varvec{f}(\varvec{X}_{i})-\mu _{R})(\varvec{f}(\varvec{X}_{i})-\mu _{R})^{T} \right) '\\ &\quad= \sum \limits _{i=1}^{N} \left( K^{\prime }(\psi (\varvec{A})) \cdot (\varvec{f}(\varvec{X}_{i})-\mu _{R})(\varvec{f}(\varvec{X}_{i})-\mu _{R})^{T}\right) , \end{aligned}$$

we obtain $\frac{\partial {\varvec{C}}_{R} (\psi (\varvec{A}))}{\partial \varvec{A}}$ as follows. In Eq. (18), $K^{\prime }(\psi (\varvec{A}))$ denotes the first derivative of Gaussian kernel function $K(\psi (\varvec{A}))= \exp (-\frac{1}{2} \Vert \psi (\varvec{A})\Vert ^{2})$ with respect to affine transformation parameters $\varvec{A}$.

The first derivative of $\varvec{K(\psi (\varvec{A}))}$ with respect to translation vector $\varvec{t}$:

$$\begin{aligned} \frac{\partial K(\psi (\varvec{A})) }{\partial t_{x}} &= \frac{\partial K(\psi (\varvec{A})) }{\partial \psi (\varvec{A})} \cdot \frac{\partial \psi (\varvec{A})}{\partial t_{x}}\\ &= \exp ^{-\psi /2} \cdot \left(-\frac{1}{2}\right) \cdot \frac{\partial \psi }{\partial t_{x}}\\ &{}= -K(\psi (\varvec{A})) \\ &\quad \times \Bigg [ \frac{a (x_{c} - ( a(x'_{i} - t_{x}) + b(y'_{i} - t_{y})) )}{h_{x}^{2}} \\ &\quad + \frac{c (y_{c} - ( c(x'_{i} - t_{x}) + d(y'_{i} - t_{y})) )}{h_{y}^{2}} \Bigg ] , \end{aligned}$$

(19)

$$\begin{aligned} \frac{\partial K(\psi (\varvec{A})) }{\partial t_{y}} &= \frac{\partial K(\psi (\varvec{A})) }{\partial \psi (\varvec{A})} \cdot \frac{\partial \psi (\varvec{A})}{\partial t_{y}}\\ &= -K(\psi (\varvec{A}))\\ &\quad \times \Bigg [ \frac{b (x_{c} - ( a(x'_{i} - t_{x}) + b(y'_{i} - t_{y})) )}{h_{x}^{2}} \\ &\quad + \frac{d (y_{c} - ( c(x'_{i} - t_{x}) + d(y'_{i} - t_{y})) )}{h_{y}^{2}} \Bigg ]. \end{aligned}$$

(20)

The first derivative of $\varvec{K(\psi (\varvec{A}))}$ with respect to rotation angle ${\varvec{\theta }}$:

$$\begin{aligned} \frac{\partial K(\psi (\varvec{A})) }{\partial \theta } &= \frac{\partial K(\psi (\varvec{A})) }{\partial \psi (\varvec{A})} \cdot \frac{\partial \psi (\varvec{A})}{\partial \theta }\\ &= -K(\psi (\varvec{A})) \\ &\quad \times \Bigg [ \frac{ x_{c} - ( a(x'_{i} - t_{x}) + b(y'_{i} - t_{y}))}{h_{x}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial a}{\partial \theta } - (y'_{i} - t_{y}) \frac{\partial b}{\partial \theta } }{h_{x}} \\ & \quad + \frac{ y_{c} - ( c(x'_{i} - t_{x}) + d(y'_{i} - t_{y}))}{h_{y}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial c}{\partial \theta } - (y'_{i} - t_{y}) \frac{\partial d}{\partial \theta } }{h_{y}} \Bigg ], \end{aligned}$$

(21)

where

$$\begin{aligned} \left\{ \begin{array}{l} \partial a /\partial \theta = -\lambda _{x} \sin \theta \cos \phi - \lambda _{y} \cos \theta \sin \phi \\ \partial b /\partial \theta = \lambda _{x} \cos \theta \cos \phi - \lambda _{y} \sin \theta \sin \phi \\ \partial c /\partial \theta = \lambda _{x} \sin \theta \sin \phi - \lambda _{y} \cos \theta \cos \phi \quad .\\ \partial d /\partial \theta = -\lambda _{x} \cos \theta \sin \phi - \lambda _{y} \sin \theta \cos \phi \end{array} \right. \end{aligned}$$

(22)

The first derivative of $\varvec{K(\psi (\varvec{A}))}$ with respect to skew direction $\varvec{\phi }$:

$$\begin{aligned} \frac{\partial K(\psi (\varvec{A})) }{\partial \phi } &= \frac{\partial K(\psi (\varvec{A})) }{\partial \psi (\varvec{A})} \cdot \frac{\partial \psi (\varvec{A})}{\partial \phi }\\ &= -K(\psi (\varvec{A})) \\ &{}\quad \times \Bigg [ \frac{ x_{c} - ( a(x'_{i} - t_{x}) + b(y'_{i} - t_{y}))}{h_{x}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial a}{\partial \phi } - (y'_{i} - t_{y}) \frac{\partial b}{\partial \phi } }{h_{x}} \\ & \quad + \frac{ y_{c} - ( c(x'_{i} - t_{x}) + d(y'_{i} - t_{y}))}{h_{y}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial c}{\partial \phi } - (y'_{i} - t_{y}) \frac{\partial d}{\partial \phi } }{h_{y}} \Bigg ], \end{aligned}$$

(23)

where

$$\begin{aligned} \left\{ \begin{array}{l} \partial a /\partial \phi = -\lambda _{x} \cos \theta \sin \phi - \lambda _{y} \sin \theta \cos \phi \\ \partial b /\partial \phi = -\lambda _{x} \sin \theta \sin \phi + \lambda _{y} \cos \theta \cos \phi \\ \partial c /\partial \phi = -\lambda _{x} \cos \theta \cos \phi + \lambda _{y} \sin \theta \sin \phi \quad .\\ \partial d /\partial \phi = -\lambda _{x} \sin \theta \cos \phi - \lambda _{y} \cos \theta \sin \phi \end{array} \right. \end{aligned}$$

(24)

The first derivative of $\varvec{K(\psi (\varvec{A}))}$ with respect to scale $\varvec{\lambda }$:

$$\begin{aligned} \frac{\partial K(\psi (\varvec{A})) }{\partial \lambda _{x}} &= \frac{\partial K(\psi (\varvec{A})) }{\partial \psi (\varvec{A})} \cdot \frac{\partial \psi (\varvec{A})}{\partial \lambda _{x}}\\ &= -K(\psi (\varvec{A})) \\ &\quad \times \Bigg [ \frac{ x_{c} - ( a(x'_{i} - t_{x}) + b(y'_{i} - t_{y}))}{h_{x}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial a}{\partial \lambda _{x}} - (y'_{i} - t_{y}) \frac{\partial b}{\partial \lambda _{x}} }{h_{x}} \\ & \quad + \frac{ y_{c} - ( c(x'_{i} - t_{x}) + d(y'_{i} - t_{y}))}{h_{y}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial c}{\partial \lambda _{x}} - (y'_{i} - t_{y}) \frac{\partial d}{\partial \lambda _{x}} }{h_{y}} \Bigg ], \end{aligned}$$

(25)

$$\begin{aligned} \frac{\partial K(\psi (\varvec{A})) }{\partial \lambda _{y}} &= \frac{\partial K(\psi (\varvec{A})) }{\partial \psi (\varvec{A})} \cdot \frac{\partial \psi (\varvec{A})}{\partial \lambda _{y}}\\ &= -K(\psi (\varvec{A})) \\ &\quad \times \Bigg [ \frac{ x_{c} - ( a(x'_{i} - t_{x}) + b(y'_{i} - t_{y}))}{h_{x}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial a}{\partial \lambda _{y}} - (y'_{i} - t_{y}) \frac{\partial b}{\partial \lambda _{y}} }{h_{x}} \\ & \quad + \frac{ y_{c} - ( c(x'_{i} - t_{x}) + d(y'_{i} - t_{y}))}{h_{y}} \\ & \quad \times \frac{ - (x'_{i} - t_{x}) \frac{\partial c}{\partial \lambda _{x}} - (y'_{i} - t_{y}) \frac{\partial d}{\partial \lambda _{y}} }{h_{y}} \Bigg ], \end{aligned}$$

(26)

where

$$\begin{aligned} \left\{ \begin{array}{l} \partial a /\partial \lambda _{x} = \cos \theta \cos \phi \\ \partial b /\partial \lambda _{x} = \sin \theta \cos \phi \\ \partial c /\partial \lambda _{x} = - \cos \theta \sin \phi \quad ,\\ \partial d /\partial \lambda _{x} = - \sin \theta \sin \phi \end{array} \right. \end{aligned}$$

(27)

and

$$\begin{aligned} \left\{ \begin{array}{l} \partial a /\partial \lambda _{y} = -\sin \theta \sin \phi \\ \partial b /\partial \lambda _{y} = \cos \theta \sin \phi \\ \partial c /\partial \lambda _{y} = - \sin \theta \cos \phi \quad .\\ \partial d /\partial \lambda _{y} = \cos \theta \cos \phi \end{array} \right. \end{aligned}$$

(28)