Skip to main content
SpringerLink
Log in

A Survey of Multi-object Video Tracking Algorithms

  • Conference paper
  • First Online:
International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018 (ATCI 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 842))

Abstract

Video multi-object tracking is one of the important research topics in the field of computer vision, which is widely used in military and civil areas. At present, the research of single object tracking algorithm is quite mature, however the research of multi-object tracking is still ongoing. This paper focuses on four important stages in the multi-object tracking process: feature extraction, detector, data association and the tracker. The feature extraction part introduces the current methods of feature extraction, as well as the merits and demerits of each method; In the stage of detection, the tracking effect of the object appearance model in specific applications is described, and then the paper analyze the multi-object tracking algorithm based on detection and tracking as well as the multi-object tracking algorithm based on deep learning; In the tracking stage, the establishment of object motion model and multi-object tracking with different tracker hybrid algorithm are introduced; During the stage of data association, the paper introduce the multi-object tracking based on energy minimization and commonly used data association algorithm, respectively. Then the current mainstream datasets and evaluation methods are introduced. Finally, the future development of the multi-object tracking is discussed and forecasted.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2411–2418. IEEE (2013)

    Google Scholar 

  2. Huang, K.Q., Chen, X.T., Kang, Y.F., Tan, T.N.: Intelligent visual surveillance: a review. Chin. J. Comput. 38, 1 (2015)

    MathSciNet  Google Scholar 

  3. Zeng, Q., Wen, G., Li, D.: Multi-target tracking by detection. In: 2016 International Conference on Audio, Language and Image Processing (ICALIP), pp. 370–374. IEEE (2016)

    Google Scholar 

  4. Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4293–4302. IEEE (2016)

    Google Scholar 

  5. Huang, K.Q., Ren, W.Q., Tan, T.N.: A review on image object classification and detection. Chin. J. Comput. 37(6), 1225–1240 (2014)

    Google Scholar 

  6. Viola, P., Jones, M. J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: Null, p. 734. IEEE (2003)

    Google Scholar 

  7. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision, pp. 702–715. Springer, Berlin (2012)

    Chapter  Google Scholar 

  8. He, S., Yang, Q., Lau, R.W., Wang, J., Yang, M.H.: Visual tracking via locality sensitive histograms. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2427–2434. IEEE (2013)

    Google Scholar 

  9. Zhang, K., Zhang, L., Yang, M.H.: Real-time compressive tracking. In: European Conference on Computer Vision, pp. 864–877. Springer, Berlin (2012)

    Chapter  Google Scholar 

  10. Chen, J., Liu, Y., Li, N., Guo, Z.: Compressive tracking based on random channel haar-like feature. In: 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP), pp. 151–154. IEEE (2015)

    Google Scholar 

  11. Possegger, H., Mauthner, T., Bischof, H.: In defense of color-based model-free tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2113–2120 (2015)

    Google Scholar 

  12. Danelljan, M., Shahbaz Khan, F., Felsberg, M., Van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, USA, 24–27 June 2014, pp. 1090–1097. IEEE Computer Society (2014)

    Google Scholar 

  13. Wang, Y., Liu, J., Wang, J., Li, Y., Lu, H.: Color names learning using convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 217–221. IEEE (2015)

    Google Scholar 

  14. Wang, Y., Wang, Q., Chen, D.: Research on target recognition based on edge features. In: 2012 5th International Congress on Image and Signal Processing (CISP), pp. 1312–1315. IEEE (2012)

    Google Scholar 

  15. Wu, J.Z., Huang, Y.D.: Image denoising algorithm based on edge feature extraction in curvelet domain. In: 2012 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), pp. 7–10. IEEE (2012)

    Google Scholar 

  16. Ojala, T., Pietikainen, M., Harwood, D.: Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, 1994. Vol. 1-Conference A: Computer Vision and Image Processing, vol. 1, pp. 582–585. IEEE (1994)

    Google Scholar 

  17. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)

    Article  MathSciNet  Google Scholar 

  18. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)

    Google Scholar 

  19. Hayakawa, Y., Oonuma, T., Kobayashi, H., Takahashi, A., Chiba, S., Fujiki, N.M.: Feature extraction of video using deep neural network. In: 2016 IEEE 15th International Conference on Cognitive Informatics and Cognitive Computing (ICCI* CC), pp. 465–470. IEEE (2016)

    Google Scholar 

  20. Zhang, H.L., Hu, S., Yang, G.: Video object tracking based on appearance models learning. J. Comput. Res. Dev. 52(1), 177–190 (2015)

    Article  Google Scholar 

  21. Supancic, J.S., Ramanan, D.: Self-paced learning for long-term tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2379–2386 (2013)

    Google Scholar 

  22. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

    Google Scholar 

  23. Girshick, R.: Fast R-CNN. In: IEEE International Conference on Computer Vision, pp. 1440–1448. IEEE Computer Society (2015)

    Google Scholar 

  24. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  25. Yuan, G.W., Chen, Z.Q., Gong, J., Xu, D., Liao, R.J., He, J.Y.: A moving object detection algorithm based on a combination of optical flow and three-frame difference. J. Chin. Comput. Syst. 34(3), 668–671 (2013)

    Google Scholar 

  26. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremer, D., Brox, T.: Flownet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)

    Google Scholar 

  27. Bing-Bing, W., Zhi-Xin, C., Jia, W., Liquan, Z.: Pedestrian detection based on the combination of HOG and background subtraction method. In: 2011 International Conference on Transportation, Mechanical, and Electrical Engineering (TMEE), pp. 527–531. IEEE (2011)

    Google Scholar 

  28. Suresh, S., Deepak, P., Chitra, K.: An efficient low cost background subtraction method to extract foreground object during human tracking. In: 2014 International Conference on Circuit, Power and Computing Technologies (ICCPCT), pp. 1432–1436. IEEE (2014)

    Google Scholar 

  29. Han, X., Gao, Y., Lu, Z., Zhang, Z., Niu, D.: Research on moving object detection algorithm based on improved three frame difference method and optical flow. In: 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC), pp. 580–584. IEEE (2015)

    Google Scholar 

  30. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: single shot multibox detector. In: European conference on computer vision, pp. 21–37. Springer, Cham (2016)

    Chapter  Google Scholar 

  31. Fard, M.K., Yazdi, M., MasnadiShirazi, M.: A block matching based method for moving object detection in active camera. In: 2013 5th Conference on Information and Knowledge Technology (IKT), pp. 443–446. IEEE (2013)

    Google Scholar 

  32. Li, W., Yao, J., Dong, T., Li, H.: Object tracking based on fragment template and multi-feature adaptive fusion. In: 2015 8th International Symposium on Computational Intelligence and Design (ISCID), vol. 2, pp. 481–484. IEEE (2015)

    Google Scholar 

  33. Zhang, J., Presti, L.L., Sclaroff, S.: Online multi-person tracking by tracker hierarchy. In: 2012 IEEE 9th International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 379–385. IEEE (2012)

    Google Scholar 

  34. Guan, Y., Chen, X., Yang, D., Wu, Y.: Multi-person tracking-by-detection with local particle filtering and global occlusion handling. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2014)

    Google Scholar 

  35. Gan, M., Cheng, Y., Wang, Y., Chen, J.: Hierarchical particle filter tracking algorithm based on multi-feature fusion. J. Syst. Eng. Electron. 27(1), 51–62 (2016)

    Google Scholar 

  36. Wang, N., Yeung, D.Y.: Learning a deep compact image representation for visual tracking. In: Advances in Neural Information Processing Systems, pp. 809–817 (2013)

    Google Scholar 

  37. Jin, Z., Liu, C.: Accelerated TLD algorithm and its application in multiple target tracking. Comput. Syst. Appl. 25(6), 196–201 (2016)

    Google Scholar 

  38. Sharma, S., Khachane, A., Motwani, D.: Real time multi-object tracking using TLD framework. In: International Conference on Inventive Computation Technologies (ICICT), vol. 2, pp. 1–6. IEEE (2016)

    Google Scholar 

  39. Lu, Y., Wu, T., Chun Zhu, S.: Online object tracking, learning and parsing with and-or graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3462–3469 (2014)

    Google Scholar 

  40. Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: Siamese CNN for robust target association. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 33–40 (2016)

    Google Scholar 

  41. Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J.: POI: multiple object tracking with high performance detection and appearance feature. In: European Conference on Computer Vision, pp. 36–42. Springer, Cham (2016)

    Google Scholar 

  42. Qi, Y., Zhang, S., Qin, L., Yao, H., Huang, Q., Lim, J., Yang, M.H.: Hedged deep tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4303–4311 (2016)

    Google Scholar 

  43. Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., Yu, N.: Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism. arXiv preprint arXiv:1708.02843 (2017)

  44. Milan, A., Rezatofighi, S.H., Dick, A.R., Reid, I.D., Schindler, K.: Online multi-target tracking using recurrent neural networks. In: AAAI, pp. 4225–4232 (2017)

    Google Scholar 

  45. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  46. Xiang, J., Zhang, G., Hou, J., Sang, N., Huang, R.: Multiple target tracking by learning feature representation and distance metric jointly. arXiv preprint arXiv:1802.03252 (2018)

  47. Munkres, J.: Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5(1), 32–38 (1957)

    Article  MathSciNet  Google Scholar 

  48. Hare, S., Golodetz, S., Saffari, A., Vineet, V., Cheng, M.M., Hicks, S.L., Torr, P.H.: Struck: structured output tracking with kernels. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2096–2109 (2016)

    Article  Google Scholar 

  49. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)

    Article  Google Scholar 

  50. Kalman, R.E., Bucy, R.S.: New results in linear filtering and prediction theory. J. Basic Eng. 83(1), 95–108 (1961)

    Article  MathSciNet  Google Scholar 

  51. Julier, S.J., Uhlmann, J.K.: New extension of the Kalman filter to nonlinear systems. In: Signal processing, sensor fusion, and target recognition VI, vol. 3068, pp. 182–194. International Society for Optics and Photonics (1997)

    Google Scholar 

  52. Wan, E.A., Van Der Merwe, R.: The unscented Kalman filter for nonlinear estimation. In: Adaptive Systems for Signal Processing, Communications, and Control Symposium 2000. AS-SPCC. The IEEE 2000, pp. 153–158. IEEE (2000)

    Google Scholar 

  53. Yang, S., Li, H.: Application of EKF and UKF in target tracking problem. In: 2016 8th International Conference on Intelligent Human–Machine Systems and Cybernetics (IHMSC), vol. 1, pp. 116–120. IEEE (2016)

    Google Scholar 

  54. Gordon, N., Ristic, B., Arulampalam, S.: Beyond the Kalman Filter: Particle Filters for Tracking Applications, vol. 3, pp. 1077–2626. Artech House, London (2004)

    MATH  Google Scholar 

  55. Kokul, T., Ramanan, A., Pinidiyaarachchi, U.A.J.: Online multi-person tracking-by-detection method using ACF and particle filter. In: 2015 IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS), pp. 529–536. IEEE (2015)

    Google Scholar 

  56. Milan, A., Leal-Taixé, L., Schindler, K., Reid, I.: Joint tracking and segmentation of multiple targets. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5397–5406. IEEE (2015)

    Google Scholar 

  57. Yu, S.I., Meng, D., Zuo, W., Hauptmann, A.: The solution path algorithm for identity-aware multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3871–3879 (2016)

    Google Scholar 

  58. Maksai, A., Wang, X., Fleuret, F., Fua, P.: Non-Markovian globally consistent multi-object tracking. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2563–2573. IEEE (2017)

    Google Scholar 

  59. Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 58–72 (2014)

    Article  Google Scholar 

  60. Milan, A., Schindler, K., Roth, S.: Multi-target tracking by discrete-continuous energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2054–2068 (2016)

    Article  Google Scholar 

  61. Wei, L., Xingwei, L.: Multiple object tracking based on energy minimization. In: 2016 4th International Symposium on Computational and Business Intelligence (ISCBI), pp. 206–211. IEEE (2016)

    Google Scholar 

  62. Singer, R.A., Stein, J.J.: An optimal tracking filter for processing sensor data of imprecisely determined origin in surveillance systems. In: 1971 IEEE Conference on Decision and Control, vol. 10, pp. 171–175. IEEE (1971)

    Google Scholar 

  63. Schulter, S., Vernaza, P., Choi, W., Chandraker, M.: Deep network flow for multi-object tracking. IEEE Conference on Computer Vision and Pattern Recognition, pp. 2730–2739. IEEE Computer Society (2017)

    Google Scholar 

  64. Kim, C., Li, F., Ciptadi, A., Rehg, J.M.: Multiple hypothesis tracking revisited. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4696–4704 (2015)

    Google Scholar 

  65. Bar-Shalom, Y., Tse, E.: Tracking in a cluttered environment with probabilistic data association. Automatica 11(5), 451–460 (1975)

    Article  Google Scholar 

  66. Cox, I.J.: A review of statistical data association techniques for motion correspondence. Int. J. Comput. Vision 10(1), 53–66 (1993)

    Article  Google Scholar 

  67. Hamid Rezatofighi, S., Milan, A., Zhang, Z., Shi, Q., Dick, A., Reid, I.: Joint probabilistic data association revisited. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3047–3055 (2015)

    Google Scholar 

  68. Shi, X., Song, Y. Q., Yang, Z., Chen, J.: Multiple target tracking under occlusions using modified Joint Probabilistic Data Association. In: 2015 IEEE International Conference on Communications (ICC), pp. 6615–6620. IEEE (2015)

    Google Scholar 

  69. Chen, S.L., Xu, Y.B., Zhu, M.: K nearest neighbor joint possibility data association algorithm. In: 2010 2nd International Conference on Information Engineering and Computer Science (ICIECS), pp. 1–4. IEEE (2010)

    Google Scholar 

  70. Son, J., Baek, M., Cho, M., Han, B.: Multi-object tracking with quadruplet convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5620–5629 (2017)

    Google Scholar 

  71. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. pp. 248–255. IEEE (2009)

    Google Scholar 

  72. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  73. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, D., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer, Cham (2014)

    Google Scholar 

  74. Kristan, M., Pflugfelder, R., Leonardis, A., Matas, J., Porikli, F., Cehovin, L., Nebehay, G., Fernandez, G., Vojir, T., Gatt, A., Khajenezhad, A.: The visual object tracking vot2013 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 98–111 (2013)

    Google Scholar 

  75. Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv preprint arXiv:1504.01942 (2015)

  76. Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: MOT16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)

  77. Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. J. Image Video Process. 2008, 1 (2008)

    Article  Google Scholar 

  78. Zhang, Z., Wu, J., Zhang, X., Zhang, C.: Multi-target, multi-camera tracking by hierarchical clustering: recent progress on DukeMTMC Project. arXiv preprint arXiv:1712.09531 (2017)

Download references

Acknowledgments

This work was supported by the Scientific Research Fund of Hunan Provincial Education Department of China (Project No. 17A007); and the Teaching Reform and Research Project of Hunan Province of China (Project No. JG1615).

Author information

Authors and Affiliations

  1. Hunan Provincial Key Laboratory of Intelligent Processing of Big Data on Transportation, Changsha University of Science and Technology, Changsha, 410114, China

    Shuren Zhou, Maolin Ke, Jia Qiu & Jin Wang

  2. School of Computer and Communication Engineering, Changsha University of Science and Technology, Changsha, 410114, China

    Shuren Zhou, Maolin Ke, Jia Qiu & Jin Wang

Authors
  1. Shuren Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maolin Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Wang .

Editor information

Editors and Affiliations

  1. Faculty of Science, Engineering and Built Environment, Deakin University Geelong, Geelong, VIC, Australia

    Jemal Abawajy

  2. Department of Information Systems and Cyber Security, The University of Texas at San Antonio, San Antonio, TX, USA

    Kim-Kwang Raymond Choo

  3. School of Computing and Mathematics, Charles Sturt University, Albury, NSW, Australia

    Rafiqul Islam

  4. Tsinghua University, Beijing Shi, China

    Zheng Xu

  5. School of Computer Science, University of Oklahoma, Norman, OK, USA

    Mohammed Atiquzzaman

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, S., Ke, M., Qiu, J., Wang, J. (2019). A Survey of Multi-object Video Tracking Algorithms. In: Abawajy, J., Choo, KK., Islam, R., Xu, Z., Atiquzzaman, M. (eds) International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018. ATCI 2018. Advances in Intelligent Systems and Computing, vol 842. Springer, Cham. https://doi.org/10.1007/978-3-319-98776-7_38

Download citation

Publish with us

Policies and ethics

Search

Navigation