Improving Global Multi-target Tracking with Local Updates

  • Anton MilanEmail author
  • Rikke Gade
  • Anthony Dick
  • Thomas B. Moeslund
  • Ian Reid
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8927)


We propose a scheme to explicitly detect and resolve ambiguous situations in multiple target tracking. During periods of uncertainty, our method applies multiple local single target trackers to hypothesise short term tracks. These tracks are combined with the tracks obtained by a global multi-target tracker, if they result in a reduction in the global cost function. Since tracking failures typically arise when targets become occluded, we propose a local data association scheme to maintain the target identities in these situations. We demonstrate a reduction of up to \(50\,\%\) in the global cost function, which in turn leads to superior performance on several challenging benchmark sequences. Additionally, we show tracking results in sports videos where poor video quality and frequent and severe occlusions between multiple players pose difficulties for state-of-the-art trackers.


Multi-target tracking Data association 

Supplementary material

Supplementary material (MP4 9,250 KB)


  1. 1.
    Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: CVPR 2008 (2008)Google Scholar
  2. 2.
    Berclaz, J., Fleuret, F., Türetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. IEEE T. Pattern Anal. Mach. Intell. 33(9), 1806–1819 (2011)CrossRefGoogle Scholar
  3. 3.
    Yang, B., Nevatia, R.: An online learned CRF model for multi-target tracking. In: CVPR 2012, pp. 2034–2041 (2012)Google Scholar
  4. 4.
    Roshan Zamir, A., Dehghan, A., Shah, M.: GMCP-Tracker: global multi-object tracking using generalized minimum clique graphs. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 343–356. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  5. 5.
    Liu, J., Carr, P., Collins, R.T., Liu, Y.: Tracking sports players with context-conditioned motion models. In: CVPR 2013, pp. 1830–1837 (2013)Google Scholar
  6. 6.
    Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. IEEE T. Pattern Anal. Mach. Intell. 36(1), 58–72 (2014)CrossRefGoogle Scholar
  7. 7.
    Vermaak, J., Doucet, A., Pérez, P.: Maintaining multi-modality through mixture tracking. In: ICCV 2003, pp. 1110–1116 (2003)Google Scholar
  8. 8.
    Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., Van Gool, L.: Robust tracking-by-detection using a detector confidence particle filter. In: ICCV 2009 (2009)Google Scholar
  9. 9.
    Wu, Y., Lim, J., Yang, M.H.: Online object tracking: A benchmark. In: CVPR 2013, pp. 2411–2418 (2013)Google Scholar
  10. 10.
    Bibby, C., Reid, I.: Real-time tracking of multiple occluding objects using level sets. In: CVPR 2010, pp. 1307–1314 (2010)Google Scholar
  11. 11.
    Mitzel, D., Horbert, E., Ess, A., Leibe, B.: Multi-person tracking with sparse detection and continuous segmentation. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 397–410. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  12. 12.
    Yan, X., Wu, X., Kakadiaris, I.A., Shah, S.K.: To track or to detect? an ensemble framework for optimal selection. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 594–607. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  13. 13.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering 82(Series D), 35–45 (1960)Google Scholar
  14. 14.
    Julier, S.J., Uhlmann, J.K.: A new extension of the kalman filter to nonlinear systems. In: International Symposium on Aerospace and Defense Sensing, Simulation and Controls, pp. 182–193 (1997)Google Scholar
  15. 15.
    Jiang, H., Fels, S., Little, J.J.: A linear programming approach for multiple object tracking. In: CVPR 2007 (2007)Google Scholar
  16. 16.
    Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR 2011 (2011)Google Scholar
  17. 17.
    Henriques, J.A., Caseiro, R., Batista, J.: Globally optimal solution to multi-object tracking with merged measurements. In: ICCV 2011 (2011)Google Scholar
  18. 18.
    Butt, A.A., Collins, R.T.: Multi-target tracking by lagrangian relaxation to min-cost network flow. In: CVPR 2013 (2013)Google Scholar
  19. 19.
    Leibe, B., Schindler, K., Van Gool, L.: Coupled detection and trajectory estimation for multi-object tracking. In: ICCV 2007 (2007)Google Scholar
  20. 20.
    Wu, Z., Thangali, A., Sclaroff, S., Betke, M.: Coupling detection and data association for multiple object tracking. In: CVPR 2012 (2012)Google Scholar
  21. 21.
    Brendel, W., Amer, M.R., Todorovic, S.: Multiobject tracking as maximum weight independent set. In: CVPR 2011 (2011)Google Scholar
  22. 22.
    Wu, Z., Kunz, T.H., Betke, M.: Efficient track linking methods for track graphs using network-flow and set-cover techniques. In: CVPR 2011 (2011)Google Scholar
  23. 23.
    Berclaz, J., Fleuret, F., Fua, P.: Multiple object tracking using flow linear programming. In: 12th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (Winter-PETS) (December 2009)Google Scholar
  24. 24.
    Milan, A., Schindler, K., Roth, S.: Detection- and trajectory-level exclusion in multiple object tracking. In: CVPR 2013 (2013)Google Scholar
  25. 25.
    Kalal, Z., Matas, J., Mikolajczyk, K.: P-N Learning: Bootstrapping binary classifiers from unlabeled data by structural constraint. In: CVPR 2010 (2010)Google Scholar
  26. 26.
    Hare, S., Saffari, A., Torr, P.H.S.: Struck: Structured output tracking with kernels. In: ICCV 2011, pp. 263–270 (2011)Google Scholar
  27. 27.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 702–715. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  28. 28.
    Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. In: CVPR 2009 (2009)Google Scholar
  29. 29.
    Li, X., Shen, C., Shi, Q., Dick, A., van den Hengel, A.: Non-sparse linear representations for visual tracking with online reservoir metric learning. In: CVPR 2012, pp. 1760–1767 (2012)Google Scholar
  30. 30.
    Izadinia, H., Saleemi, I., Li, W., Shah, M.: (MP)\(^\text{2 }\)T: Multiple people multiple parts tracker. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 100–114. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  31. 31.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE T. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  32. 32.
    Zhang, L., van der Maaten, L.: Structure preserving object tracking. In: CVPR 2013, pp. 1838–1845 (2013)Google Scholar
  33. 33.
    Delong, A., Osokin, A., Isack, H.N., Boykov, Y.: Fast approximate energy minimization with label costs. Int. J. Comput. Vision 96(1), 1–27 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  34. 34.
    Ferryman, J., Shahrokni, A.: PETS2009: Dataset and challenge. In: 11th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS) (December 2009)Google Scholar
  35. 35.
    Andriluka, M., Roth, S., Schiele, B.: Monocular 3D pose estimation and tracking by detection. In: CVPR 2010 (2010)Google Scholar
  36. 36.
    Milan, A., Schindler, K., Roth, S.: Challenges of ground truth evaluation of multi-target tracking. In: 2013 IEEE CVPR Workshops (CVPRW), pp. 735–742 (June 2013)Google Scholar
  37. 37.
    Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: The CLEAR MOT metrics. Image and Video Processing 2008(1), 1–10 (2008)CrossRefGoogle Scholar
  38. 38.
    Li, Y., Huang, C., Nevatia, R.: Learning to associate: Hybridboosted multi-target tracker for crowded scene. In: CVPR 2009 (2009)Google Scholar
  39. 39.
    Dollár, P., Appel, R., Belongie, S., Perona, P.: Fast feature pyramids for object detection. IEEE T. Pattern Anal. Mach. Intell. (2014) (to appear)Google Scholar
  40. 40.
    Dicle, C., Sznaier, M., Camps, O.: The way they move: Tracking multiple targets with similar appearance. In: ICCV 2013 (2013)Google Scholar
  41. 41.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR 2005, pp. 886–893 (2005)Google Scholar
  42. 42.
    Walk, S., Majer, N., Schindler, K., Schiele, B.: New features and insights for pedestrian detection. In: CVPR 2010 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Anton Milan
    • 1
    Email author
  • Rikke Gade
    • 2
  • Anthony Dick
    • 1
  • Thomas B. Moeslund
    • 2
  • Ian Reid
    • 1
  1. 1.University of AdelaideAdelaideAustralia
  2. 2.Aalborg UniversityAalborgDenmark

Personalised recommendations