Combining Siamese Network and Correlation Filter for Complementary Object Tracking

Abstract

Fully-Convolutional Siamese Network (SiamFC) is convolutional neural networks (CNNs) model-based tracking method. This method learns a similarity map from the cross-correlation between the feature representations of the search image and the target image extracted using CNNs, and tracks based on it. The tracking performance of SiamFC tends to degrade when there are similar distractors to the object or when the target object is deformed. On the other hand, recent object tracking methods using the correlation filter (CF) drift under some scenarios such as fast motion and complete occlusion. The analysis showed that although these two approaches have very different structures, they tend to have complementary characteristics. In this work, we propose a complementary tracking framework that parallel connects SiamFC with the CF-based tracker. In the proposed framework, to detect tracking failures, we evaluate the response map output from SiamFC using the confidence score defined in this paper. When a tracking failure is detected, the CF-based tracker provides relative correlated correction. Experiments on the OTB2015 show that our tracker obtains up to more than 5.7%/5.5% (precision score/success score) relative improvements over the original SiamFC and CF-based tracker on the OTB2015, and competitive performance with advanced trackers.