A target response adaptive correlation filter tracker with spatial attention

Abstract

In recent years, many efficient and accurate algorithms have been proposed in the field of target tracking. The existing correlation filter(CF) based tracking expands the sample through cyclic shift and uses the position of the maximum response map as the most likely position of the target. Since the cyclic shift sample is not the real sample, the target response may also occur in the non-target region, which not only causes the boundary effect problem but also results in the positioning error. To train the filter with the wrong position will lead to the tracking drift. This paper proposes a target response adaptive correlation filter tracker with spatial attention to solve the above problems. Firstly, more useful feature information can be learned by making full use of the context information of the target area, and spatial attention mechanisms can be introduced to suppress the background information and reduce the unnecessary boundary effect. Secondly, the dynamic change of the target response is captured, and the most reliable response map is selected when the interference response map appears, to reduce the probability of dispositioning and train the more recognizable filter. Extensive experimental results on popular benchmarks(OTB2013 OTB100 and VOT2016) demonstrate that the proposed tracker performs favorably against other state-of-the-art tracking methods, with real-time performance.

Appendix

The derivation of (6) and (7) is shown as follows:

Equation (5) can be rewritten in terms of z with ${z}^{T} = \left [ {w^{T} y^{T} } \right ]$:

$$ \begin{array}{@{}rcl@{}} f(z) &=& ||\left[{A_{0}- I} \right]z||_{2}^{2} + \lambda_{1} ||\left[ {\begin{array}{ll} S \end{array}} \right]z||_{2}^{2} + \lambda_{2} ||\left[ {\begin{array}{ll} 0& I \end{array}} \right]z - y_{0} ||_{2}^{2} \\ &&+ \lambda_{3} \sum\limits_{i = 1}^{k} {||A_{i} } \left[ {\begin{array}{ll} I &0 \end{array}} \right]z||_{2}^{2} \end{array} $$

(13)

Where w ∈ Rⁿ, y ∈ Rⁿ, y₀ ∈ Rⁿ, z ∈ R²ⁿ,then:

$$ \begin{array}{@{}rcl@{}} \nabla_{z} f(z) &=& \left[ {\begin{array}{ll} {A^{T} A} & { - A^{T} } \\ { - A} & I \end{array}} \right]z + \lambda_{1} \left[ {\begin{array}{*{20}c} {S^{T} S} & 0 \\ 0 & 0 \end{array}} \right]z + \lambda_{2} \left[ {\begin{array}{*{20}c} 0 & 0 \\ 0 & I \end{array}} \right]z \\ &&- \lambda_{2} \left[ {\begin{array}{*{20}c} 0 \\ I \end{array}} \right]y_{0} + \lambda_{3} \sum\limits_{i = 1}^{k} {\left[ {\begin{array}{*{20}c} {{A_{i}^{T}} A_{i} } & 0 \\ 0 & 0 \end{array}} \right]} z = 0 \end{array} $$

so

$$ \nabla_{z} f(z) = \left[ {\begin{array}{*{20}c} {A^{T} A + \lambda_{1} {S^{T} S} + \lambda_{3} \sum\limits_{i = 1}^{k} {{A_{i}^{T}} A_{i} } } & { - A^{T} } \\ { - A} & {(1 + \lambda_{2} )I} \end{array}} \right]z = \lambda_{2} \left[ {\begin{array}{*{20}c} 0 \\ I \end{array}} \right]y_{0} $$

$$ \begin{array}{@{}rcl@{}} \begin{array}{ll} \left[ {\begin{array}{*{20}c} F & 0 \\ 0 & F \end{array}} \right]\left[ {\begin{array}{*{20}c} {diag(\hat a_{0} \odot \hat a_{0}^ * + {\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )} & { - diag(\hat a_{0}^ * )} \\ { - diag(\hat a_{0} )} & {diag(1 + \lambda_{2} )} \end{array}} \right] \\ \left[ {\begin{array}{*{20}c} {F^{H} } & 0 \\ 0 & {F^{H} } \end{array}} \right]z = \lambda_{2} \left[ {\begin{array}{*{20}c} 0 \\ {F^{H} } \end{array}} \right]y_{0} \end{array} \end{array} $$

$$ \left[ {\begin{array}{*{20}c} {diag(\hat a_{0} \odot \hat a_{0}^ * + {\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )} & { - diag(\hat a_{0}^ * )} \\ {\! - \!diag(\hat a_{0} )} & {diag(1 + \lambda_{2} )} \end{array}} \right]\left[ {\begin{array}{*{20}c} {\hat w^{*} } \\ {\hat y^{*} } \end{array}} \right] \!= \! \lambda_{2} \left[ {\begin{array}{*{20}c} 0 \\ {F^{H} } \end{array}} \right]y_{0} $$

$$ \left[ {\begin{array}{*{20}c} {\hat w^{*} } \\ {\hat y^{*} } \end{array}} \right] \!= \! \lambda_{2} \left[ {\begin{array}{*{20}c} {diag(\hat a_{0} \odot \hat a_{0}^ * \! +\! {\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )} & { - diag(\hat a_{0}^ * )} \\ { - diag(\hat a_{0} )} & {diag(1 + \lambda_{2} )} \end{array}} \right]^{ - 1} \left[ {\begin{array}{*{20}c} 0 \\ {F^{H} } \end{array}} \right]y_{0} $$

Note that the inverse lemma states:

$$ \left[ {\begin{array}{*{20}c} B & N \\ V & C \end{array}} \right]^{ - 1} \!= \!\left[ {\begin{array}{*{20}c} {(B\! -\! NC^{\! -\! 1} V)^{ - 1} } & {(B - NC^{ - 1} V)^{ - 1} NC^{ - 1} } \\ { - C^{\!- \!1} V(B - NC^{ - 1} V)^{ - 1} } & {C^{\!- \!1} V(B - NC^{ - 1} V)^{ - 1} NC^{\! - \!1} { + }}C^{\!- \!1} \end{array}} \right] $$

(14)

Then:

$$ B - NC^{- 1} V{ = }diag(\hat a_{0} \odot \hat a_{0}^ * + {\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } ) - diag(\hat a_{0}^ * )diag^{- 1} (1 + \lambda_{2} )diag(\hat a_{0} ) $$

$$ (B - NC^{- 1} V)^{- 1} = diag\left( {\frac{{1 + \lambda_{2} }}{{\lambda_{2} (\hat a_{0} \odot \hat a_{0}^ * ) + (1 + \lambda_{2} )({\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )}}} \right) $$

$$ - (B\! -\! NC^{- 1} V)^{- 1} NC^{- 1} = diag\left( {\frac{{\hat a_{0}^ * }}{{\lambda_{2} (\hat a_{0} \odot \hat a_{0}^ * ) + (1 + \lambda_{2} )({\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )}}} \right) $$

Since:

$$ \hat w^{*} = - \lambda_{2} (B - NC^{- 1} V)^{- 1} NC^{- 1} F^{H} y_{0} $$

(15)

Then:

$$ \hat w^{*} = \frac{{\lambda_{2} (\hat a_{0}^ * \odot \hat y_{0}^ * )}}{{\lambda_{2} (\hat a_{0} \odot \hat a_{0}^ * + (1 + \lambda_{2} )({\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )}} $$

⇒

$$ \hat w = \frac{{\lambda_{2} (\hat a_{0} \odot \hat y_{0} )}}{{\lambda_{2} (\hat a_{0} \odot \hat a_{0}^ * ) + (1 + \lambda_{2} )({\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )}} $$

Since:

$$ w = {A_{0}^{T}} \alpha $$

(16)

Then:

$$ \hat \alpha = \frac{{\lambda_{2} \hat y_{0} }}{{\lambda_{2} (\hat a_{0} \odot \hat a_{0}^ * + (1 + \lambda_{2} )({\lambda_{1} \hat s \odot \hat s^ * } + \lambda_{3} \sum\limits_{i = 1}^{k} {\hat a_{i} \odot \hat a_{i}^ * } )}} $$