Skip to main content
SpringerLink
Log in

Multi-object tracking via MHT with multiple information fusion in surveillance video

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Tracking multiple objects is critical to automatic video content analysis and virtual reality. The major problem is how to solve data association problem when ambiguous measurements are caused by objects in close proximity. To tackle this problem, we propose a multiple information fusion-based multiple hypotheses tracking algorithm integrated with appearance feature, local motion pattern feature and repulsion–inertia model for multi-object tracking. Appearance model based on HSV–local binary patterns histogram and local motion pattern based on optical flow are adopted to describe objects. A likelihood calculation framework is proposed to incorporate the similarities of appearance, dynamic process and local motion pattern. To consider the changes in appearance and motion pattern over time, we make use of an effective template updating strategy for each object. In addition, a repulsion–inertia model is adopted to explore more useful information from ambiguous detections. Experimental results show that the proposed approach generates better trajectories with less missing objects and identity switches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Hofmann, M., Rigoll, G., Huang, T.: Dense spatio-temporal motion segmentation for tracking multiple self-occluding people. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 9–14 (2010)

  2. Zhang, T., Liu, J., Liu, S., Ouyang, Y., Lu, H.: Boosted exemplar learning for human action recognition. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), pp. 538–545(2009)

  3. Wang, C., Gorce, M., Paragios, N.: Segmentation, ordering and multi-object tracking using graphical models. In: 2009 IEEE 12th International Conference on Computer Vision (2009)

  4. Xing, J., Ai, H., Lao, S.: Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1200–1207 (2009)

  5. Yu, Q., Medioni, G.: Multiple-target tracking by spatio-temporal Monte Carlo Markov chain data association. TPAMI 31(12), 2196–2210 (2009)

    Article  Google Scholar 

  6. Popp, R.L., Pattipati, K.R., Bar-Shalom, Y.: Dynamically adaptable m-best 2-D assignment algorithm and multilevel parallelization. IEEE Trans. Aerosp. Electron. Syst. 35(4), 1145–1160 (1999)

    Article  Google Scholar 

  7. Bar-Shalom, Y., Daum, F., Huang, J.: The probabilistic data association filter. IEEE Control Syst. 29(6), 82–100 (2009)

    Article  MathSciNet  Google Scholar 

  8. Knan, Z., Balch, T., Dellaert, F.: MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE Trans. Pattern Anal. Mach. Intell. 27(11), 1805–1819 (2005)

    Article  Google Scholar 

  9. Qu, W., Schonfeld, D., Mohamed, M.: Real time distributed multi-object tracking using multiple interactive trackers and a magnetic-inertia potential model. IEEE Trans. Multimed. 9(3), 511–519 (2007)

    Article  Google Scholar 

  10. Breitenstein, M.D., Reichlin, F., et al.: Robust tracking-by-detection using a detector confidence particle filter. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1515–1522 (2009)

  11. Reid, D.: An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 24(6), 843–854 (1979)

    Article  Google Scholar 

  12. Cox, I., Hingorani, S.: An efficient implementation of Reid’s multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. 18(2), 138–150 (1996)

    Article  Google Scholar 

  13. Miller, M., Stone, H., Cox, I.: Optimizing Murty’s ranked assignment method. IEEE Trans. Aerosp. Electron. Syst. 33(3), 851–862 (1997)

    Article  Google Scholar 

  14. Ryoo, M., Aggarwal, J.: Observe-and-explain: a new approach for multiple hypotheses tracking of humans and objects. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1–8 (2008)

  15. Oh, S., Russell, S., Sastry, S.: Markov Chain Monte Carlo data association for multi-target tracking. IEEE Trans. Autom. Control 54(3), 481–497 (2009)

    Article  MathSciNet  Google Scholar 

  16. Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp. 3457–3464 (2011)

  17. Shitrit, H., Berclaz, J., Fleuret, F., et al.: Tracking multiple people under global appearance constraints. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 137–144 (2011)

  18. Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1–8 (2008)

  19. Pirsiavash, H., Ramanan, D., Fowlkes, C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1201–1208 (2011)

  20. Brendel, W., Amer, M., Todorovic, S.: Multiobject tracking as maximum weight independent set. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1273–1280 (2011)

  21. Yang, B., Huang, C., Nevatia, R.: Learning affinities and dependencies for multi-target tracking using a CRF model. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1233–1240 (2011)

  22. Ying, L., Xu, C.S., Guo, W.: Extended MHT algorithm for multiple object tracking. In: Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, pp. 75–79 (2012)

  23. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  24. Black, M., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)

    Article  Google Scholar 

  25. Zhang, T., Liu, S., Xu, C., Lu, H.: Mining semantic context information for intelligent video surveillance of traffic scenes. In: IEEE transactions on industrial informatics (2012)

  26. Tran, D., Sorokin, A.: Human activity recognition with metric learning. In: Computer vision—ECCV 2008. Springer, Berlin/Heidelberg, pp. 548–561 (2008)

  27. Liu, C.: Classifier combination based on confidence transformation. Pattern Recogn. 38(1), 11–28 (2005)

    Article  MATH  Google Scholar 

  28. Zhang, T.Z., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via structured multi-task sparse learning. Int. J. Comput. Vis. 101(2), 367–383 (2012)

    Article  MathSciNet  Google Scholar 

  29. Zhang, T.Z., Ghanem, B., Liu, S., Ahuja, N.: Robust visual tracking via multi-task sparse learning. In: IEEE International Conference on Computer Vision and Pattern Recognition (2012)

  30. Zhang, T.Z., Ghanem, B., Liu, S., Ahuja, N.: Low-rank sparse learning for robust visual tracking. In: European Conference on Computer Vision (2012)

  31. Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. EURASIP J Image Video Process (2008)

  32. Zha, Zheng-Jun, Wang, Meng, Zheng, Yan-Tao, Yang, Yi, Hong, Richang, Chua, Tat-Seng: Interactive video indexing with statistical active learning. IEEE Trans. Multimed. 14(1), 17–27 (2012)

    Article  Google Scholar 

  33. Zha, Z.-J., Yang, L., Mei, T., Wang, M., Wang, Z., Chua, T.-S., Hua, X.-S.: Visual query suggestion: towards capturing user intent in internet image search. ACM Trans. Multimed. Comput. Commun. Appl (TOMMCAP) 6(3), article no 13 (2010)

    Google Scholar 

  34. Zha, Z-J., Hua, X-S., Mei, T., Wang, J., Qi, G-J., Wang, Z.: Joint multi-label multi-instance l earning for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)

Download references

Acknowledgments

This work is supported by National Natural Science Foundation of China (61303173).

Author information

Authors and Affiliations

  1. National Lab of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Long Ying, Tianzhu Zhang & Changsheng Xu

Authors
  1. Long Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianzhu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changsheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Long Ying.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ying, L., Zhang, T. & Xu, C. Multi-object tracking via MHT with multiple information fusion in surveillance video. Multimedia Systems 21, 313–326 (2015). https://doi.org/10.1007/s00530-014-0361-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0361-5

Keywords

Search

Navigation