Online Multi-Person Tracking Based on Metric Learning

  • Changyong Yu
  • Min Yang
  • Yanmei Dong
  • Mingtao PeiEmail author
  • Yunde Jia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9916)


The correct associations of detections and tracklets are the key to online multi-person tracking. Good appearance models can guide data association and play an important role in the association. In this paper, we construct a discriminative appearance model by using metric learning which can obtain accurate appearance affinities with human appearance variations. The novel appearance model can significantly guide data association. Furthermore, the model is learned incrementally according to the association results and its parameters are automatically updated to be suitable for the next online tracking. Based on an online tracking-by-detection framework, our method achieves reliable tracking of multiple persons even in complex scenes. Our experimental evaluation on publicly available data sets shows that the proposed online multi-person tracking method works well.


Online tracking Metric learning Multi-person tracking Appearance model 



This work was supported in part by the Natural Science Foundation of China (NSFC) under Grant No. 61472038 and No. 61375044, and Beijing Key Laboratory of Advanced Information Science and Network Technology (No. XDXX1601).


  1. 1.
    Kuo, C.H., Nevatia, R.: How does person identity recognition help multi-person tracking? In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1217–1224. IEEE (2011)Google Scholar
  2. 2.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75(2), 247–266 (2007)CrossRefGoogle Scholar
  3. 3.
    Poiesi, F., Mazzon, R., Cavallaro, A.: Multi-target tracking on confidence maps: an application to people tracking. Comput. Vis. Image Underst. 117(10), 1257–1272 (2013)CrossRefGoogle Scholar
  4. 4.
    Davis, J.V., Kulis, B., Jain, P., et al.: Information-theoretic metric learning. In: Proceedings of the 24th International Conference on Machine learning, pp. 209–216. ACM (2007)Google Scholar
  5. 5.
    Leal-Taix, L., Milan, A., Reid, I., et al.: Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv preprint arXiv:1504.01942 (2015)
  6. 6.
    Ferryman, J., Shahrokni, A.: PETS2009: dataset and challenge. In: 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS-Winter), pp. 1–6. IEEE (2009)Google Scholar
  7. 7.
    Andriluka, M., Roth, S., Schiele, B.: Monocular 3D pose estimation and tracking by detection. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 623–630. IEEE (2010)Google Scholar
  8. 8.
    Ess, A., Leibe, B., Gool, L.V.: Depth and appearance for mobile scene analysis. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–8. IEEE (2007)Google Scholar
  9. 9.
    Bae, S.H., Yoon, K.J.: Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1218–1225. IEEE (2014)Google Scholar
  10. 10.
    Geiger, A., Lauer, M., Wojek, C., et al.: 3D traffic scene understanding from movable platforms. IEEE Trans. Pattern Anal. Mach. Intell. 36(5), 1012–1025 (2014)CrossRefGoogle Scholar
  11. 11.
    Yang, B., Nevatia, R.: Multi-target tracking by online learning of non-linear motion patterns, robust appearance models. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1918–1925. IEEE (2012)Google Scholar
  12. 12.
    Breitenstein, M.D., Reichlin, F., Leibe, B., et al.: Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1820–1833 (2011)CrossRefGoogle Scholar
  13. 13.
    Wu, Y., Ma, B., Yang, M., et al.: Metric learning based structural appearance model for robust visual tracking. IEEE Trans. Circuits Syst. Video Technol. 24(5), 865–877 (2014)CrossRefGoogle Scholar
  14. 14.
    Yoon, J.H., Yang, M.H., Lim, J., et al.: Bayesian multi-object tracking using motion context from multiple objects. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 33–40. IEEE (2015)Google Scholar
  15. 15.
    Yang, M., Jia, Y.: Temporal dynamic appearance modeling for online multi-person tracking. In: Computer Vision and Image Understanding (2016)Google Scholar
  16. 16.
    Dicle, C., Camps, O., Sznaier, M.: The way they move: tracking multiple targets with similar appearance. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2304–2311 (2013)Google Scholar
  17. 17.
    Solera, F., Calderara, S., Cucchiara, R.: Learning to divide and conquer for online multi-target tracking. In: International Conference on Computer Vision (ICCV), pp. 4373–4381 (2015)Google Scholar
  18. 18.
    Possegger, H., Mauthner, T., Roth, P., et al.: Occlusion geodesics for online multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1306–1313 (2014)Google Scholar
  19. 19.
    Dollr, P., Appel, R., Belongie, S., et al.: Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1532–1545 (2014)CrossRefGoogle Scholar
  20. 20.
    Yang, M., Pei, M.T., Wu, Y.W., et al.: Learning online structural appearance model for robust object tracking. Sci. China Inf. Sci. 58(3), 1–14 (2015)CrossRefGoogle Scholar
  21. 21.
    Yang, M., Pei, M., Shen, J., Jia, Y.: Robust online multi-object tracking by maximum a posteriori estimation with sequential trajectory prior. In: Arik, S., Huang, T., Lai, W.K., Liu, Q. (eds.) ICONIP 2015. LNCS, vol. 9489, pp. 623–633. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-26532-2_69 CrossRefGoogle Scholar
  22. 22.
    Wang, B., Wang, G., Chan, K., et al.: Tracklet association with online target-specific metric learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1234–1241 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Changyong Yu
    • 1
  • Min Yang
    • 1
  • Yanmei Dong
    • 1
  • Mingtao Pei
    • 1
    Email author
  • Yunde Jia
    • 1
  1. 1.Beijing Laboratory of Intelligent Information Technology, School of Computer ScienceBeijing Institute of TechnologyBeijingPeople’s Republic of China

Personalised recommendations