Object Tracking by Incremental Structural Learning of Deformable Parts

Abstract

Online object tracking in various scenarios remains a challenging problem as it entails a contradiction between distinguishing target from background and learning target appearance in scene. In this paper, we attempt to address this problem from three different perspectives. To prompt the descriptiveness of appearance model, we introduce structural model from object detection field into tracking tasks, tracing both target as well as its deformable parts simultaneously. To prompt the robustness of tracking, we propose a logistics regression-based voting method to exclude the influence of occluded parts in tracking results and model update. Finally, we propose an online method to incrementally learn structural appearance model with the samples filtered by occlusion handling mechanism. Empirical results demonstrate that the proposed tracking framework outperforms other leading methods especially on challenging object tracking tasks.

Appendix: Derivation of DCRF on DPMs

According to the Hammersley–Clifford theorem, the posterior probability of random field $s_t$ at time t in Eq. (5) can be given by a Gibbs distribution [18] as

$$\begin{aligned} p(s_t|o_{1:t})\propto \exp \Bigg \{\sum _{x\in {X}}\Bigg [V_x(s_t(x)|o_{1:t}(x)) +\sum _{y\in {N_x}}V_{x,y}(s_t(x),s_t(y))\Bigg ]\Bigg \}. \end{aligned}$$

(17)

Since in the context of DPMs, we only account part deformation as unidirectional pairwise potential $V_{x,y}(s_t(x),s_t(y))$. The prior knowledge from previous frame can be factorized to each single vertex directly. With similar conditional independence assumption in [31], the observation model $p(o_{t+1}|s_{t+1})$ in Eq. (5) can also be evaluated by unary potentials on vertices:

$$\begin{aligned} p(o_{t+1}|s_{t+1})=\prod _{x\in {X}}p(o_{t+1}(x)|s_{t+1}(x))\propto \exp \big (\sum _{x\in {X}}V_x(o_{t+1}(x)|s_{t+1}(x))\big ). \end{aligned}$$

(18)

The state transition probability $p(s_{t+1}|s_t)$ in Eq. (5) consists of both temporal pairwise potentials and spatial pairwise potentials:

$$\begin{aligned} p(s_{t+1}|s_t)\propto \exp \big \{\sum _{x\in {X}}\big [V_x(s_{t+1}(x)|s_t(M_x)) +\sum _{y\in {N_x}}V_{x,y}(s_{t+1}(x),s_{t+1}(y))\big ]\big \}\nonumber \\ \end{aligned}$$

(19)

where potential $V_x(s_{t+1}(x)|s_t(M_x))$ actually denotes the mean of pairwise potentials between x and its temporal neighboring vertices:

$$\begin{aligned} V_x(s_{t+1}(x)|s_t(M_x))=\frac{1}{|M_x|}\sum _{y\in {M_x}}V_x(s_{t+1}(x)|s_t(y)). \end{aligned}$$

(20)

Combining Eqs. (17), (18) and (19) into Eq. (5), the probability $p(s_{t+1}|o_{1:t+1})$ becomes

$$\begin{aligned} p(s_{t+1}|o_{1:t+1})\propto & {} \exp \big (\sum _{x\in {X}}V_x(o_{t+1}(x)|s_{t+1}(x))\big ) \cdot \sum _{s_t}\prod _{x\in {x}} \exp \big [V_x(s_{t+1}(x)|s_t(M_x))\nonumber \\&+\sum _{y\in {N_x}}V_{x,y}(s_{t+1}(x),s_{t+1}(y))\big ]p(s_t(x)|o_{1:t}(x))\nonumber \\= & {} \prod _{x\in {X}}\exp \big [V_x(o_{t+1}(x)|s_{t+1}(x))+\sum _{y\in {N_x}}V_{x,y} (s_{t+1}(x),s_{t+1}(y))\big ]\nonumber \\&\cdot \sum _{s_t(x)}\exp \big [V_x(s_{t+1}(x)|s_t(M_x))\big ]p(s_t(x)|o_{1:t}(x)). \end{aligned}$$

(21)

Using Jensen’s inequality, we can evaluate $p(s_{t+1}|o_{1:t+1})$ by its lower bound:

$$\begin{aligned} p(s_{t+1}|o_{1:t+1})\approx & {} \prod _{x\in {X}}\exp \big [V_x(o_{t+1}(x)|s_{t+1}(x)) +\sum _{y\in {N_x}}V_{x,y}(s_{t+1}(x),s_{t+1}(y))\big ]\nonumber \\&\cdot \exp \Big \{\sum _{s_t(x)}p(s_t(x)|o_{1:t}(x))\big [\frac{1}{|M_x|} \sum _{y\in {M_x}}V_x(s_{t+1}(x)|s_t(y))\big ]\Big \}.\nonumber \\ \end{aligned}$$

(22)

Here we only consider corresponding vertex at last frame as temporal neighbor to current vertex. Equation (22) can be simply rewritten as Eq. (6).