Abstract
Scene texts in video are not fixed in color, size, format and are easily confused with the background, which imposes significant challenges in video scene text tracking. The trajectories are often be fragmented caused by these. Most tracking methods focus on the matching of the appearance features and the temporal information across frames, treating each text as a separate object. However, the relations among all texts are also important cues. In this paper, we propose a novel online video scene text tracking approach with the spatial-temporal relation module utilizing multiple cues, i.e. appearance, geometry and temporal. The spatial-temporal relation module enhances appearance features by modeling the relations between texts with each other in the same frame, which can avoid the influence of bad detection results, and track text stably and consistently. We achieved more tracked texts and more complete trajectories on IC15 with the spatial-temporal relation module.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Bergmann, P., Meinhardt, T., Leal-Taixe, L.: Tracking without bells and whistles. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 941–951 (2019)
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. EURASIP J. Image Video Process. 2008, 1–10 (2008)
Cheng, Z., et al.: Free: a fast and robust end-to-end video text spotter. IEEE Trans. Image Process. 30, 822–837 (2020)
Chu, P., Ling, H.: Famnet: joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6172–6181 (2019)
Goto, H., Tanaka, M.: Text-tracking wearable camera system for the blind. In: 2009 10th International Conference on Document Analysis and Recognition. pp. 141–145. IEEE (2009)
Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision, pp. 702–715. Springer (2012)
Hu, H., Gu, J., Zhang, Z., Dai, J., Wei, Y.: Relation networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3588–3597 (2018)
Huang, W., Shivakumara, P., Tan, C.L.: Detecting moving text in video using temporal information. In: 2008 19th International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)
Karatzas, D., et al.: Icdar 2015 competition on robust reading. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1156–1160. IEEE (2015)
Karatzas, D., et al.: Icdar 2013 robust reading competition. In: 2013 12th International Conference on Document Analysis and Recognition, pp. 1484–1493. IEEE (2013)
Li, Y., Huang, C., Nevatia, R.: Learning to associate: hybridboosted multi-target tracker for crowded scene. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2953–2960. IEEE (2009)
Liao, M., Pang, G., Huang, J., Hassner, T., Bai, X.: Mask textspotter v3: segmentation proposal network for robust scene text spotting. arXiv preprint arXiv:2007.09482 (2020)
Liao, M., Wan, Z., Yao, C., Chen, K., Bai, X.: Real-time scene text detection with differentiable binarization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11474–11481 (2020)
Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Kim, T.K.: Multiple object tracking: a literature review. Artificial Intelligence, p. 103448 (2020)
Na, Y., Wen, D.: An effective video text tracking algorithm based on sift feature and geometric constraint. In: Pacific-Rim Conference on Multimedia, pp. 392–403. Springer (2010)
Pang, B., Li, Y., Zhang, Y., Li, M., Lu, C.: Tubetk: adopting tubes to track multi-object in a one-step training model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6308–6318 (2020)
Peng, J., et al.: Chained-tracker: chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking. In: European Conference on Computer Vision, pp. 145–161. Springer (2020)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497 (2015)
Tian, S., Pei, W.Y., Zuo, Z.Y., Yin, X.C.: Scene text detection in video by learning locally and globally. In: IJCAI, pp. 2647–2653 (2016)
Wang, L., Wang, Y., Shan, S., Su, F.: Scene text detection and tracking in video with background cues. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 160–168 (2018)
Wang, X., Feng, X., Xia, Z.: Scene video text tracking based on hybrid deep text detection and layout constraint. Neurocomputing 363, 223–235 (2019)
Wang, Z., Zheng, L., Liu, Y., Wang, S.: Towards real-time multi-object tracking. arXiv preprint arXiv:1909.12605 2(3), 4 (2019)
Wu, J., Cao, J., Song, L., Wang, Y., Yang, M., Yuan, J.: Track to detect and segment: an online multi-object tracker. arXiv preprint arXiv:2103.08808 (2021)
Wu, L., Shivakumara, P., Lu, T., Tan, C.L.: A new technique for multi-oriented scene text line detection and tracking in video. IEEE Trans. Multimed. 17(8), 1137–1152 (2015)
Xu, Y., et al.: Gliding vertex on the horizontal bounding box for multi-oriented object detection. IEEE Trans. Pattern Analysis Mach. Intell. 43, 1452–1459 (2020)
Yang, X.H., He, W., Yin, F., Liu, C.L.: A unified video text detection method with network flow. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 331–336. IEEE (2017)
Yin, J., Wang, W., Meng, Q., Yang, R., Shen, J.: A unified object motion and affinity model for online multi-object tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6768–6777 (2020)
Yin, X.C., Pei, W.Y., Zhang, J., Hao, H.W.: Multi-orientation scene text detection with adaptive clustering. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1930–1937 (2015)
Yin, X.C., Zuo, Z.Y., Tian, S., Liu, C.L.: Text detection, tracking and recognition in video: a comprehensive survey. IEEE Trans. Image Process. 25(6), 2752–2773 (2016)
Yu, H., Huang, Y., Pi, L., Zhang, C., Li, X., Wang, L.: End-to-end video text detection with online tracking. Pattern Recogn. 113, 107791 (2021)
Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., Yang, M.H.: Online multi-object tracking with dual matching attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 366–382 (2018)
Zuo, Z.Y., Tian, S., Pei, W.Y., Yin, X.C.: Multi-strategy tracking based text detection in scene videos. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 66–70. IEEE (2015)
Acknowledgement
The research is supported by National Key Research and Development Program of China (2020AAA09701), National Natural Science Foundation of China (61806017, 62006018) and Fundamental Research Funds for the Central Universities (FRF-NP-20-02).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xiu, Y., Zhou, HY., Tian, S., Yin, XC. (2021). Online Scene Text Tracking with Spatial-Temporal Relation. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-87361-5_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87360-8
Online ISBN: 978-3-030-87361-5
eBook Packages: Computer ScienceComputer Science (R0)