Advertisement

Performance Measures and a Data Set for Multi-target, Multi-camera Tracking

  • Ergys RistaniEmail author
  • Francesco Solera
  • Roger Zou
  • Rita Cucchiara
  • Carlo Tomasi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9914)

Abstract

To help accelerate progress in multi-target, multi-camera tracking systems, we present (i) a new pair of precision-recall measures of performance that treats errors of all types uniformly and emphasizes correct identification over sources of error; (ii) the largest fully-annotated and calibrated data set to date with more than 2 million frames of 1080 p, 60 fps video taken by 8 cameras observing more than 2,700 identities over 85 min; and (iii) a reference software system as a comparison baseline. We show that (i) our measures properly account for bottom-line identity match performance in the multi-camera setting; (ii) our data set poses realistic challenges to current trackers; and (iii) the performance of our system is comparable to the state of the art.

Keywords

Performance evaluation Multi camera tracking Identity management Multi camera data set Large scale data set 

References

  1. 1.
    Ristani, E., Tomasi, C.: Tracking multiple people online and in real time. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9007, pp. 444–459. Springer, Heidelberg (2015)Google Scholar
  2. 2.
    Cao, L., Chen, W., Chen, X., Zheng, S., Huang, K.: An equalised global graphical model-based approach for multi-camera object tracking [cs]. arXiv:11502.03532, February 2015
  3. 3.
    Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. EURASIP J. Image Video Process. 246309, 1–10 (2008)CrossRefGoogle Scholar
  4. 4.
    Wu, B., Nevatia, R.: Tracking of multiple, partially occluded humans based on static body part detection. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 951–958. IEEE (2006)Google Scholar
  5. 5.
    Milan, A., Schindler, K., Roth, S.: Challenges of ground truth evaluation of multi-target tracking. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 735–742. IEEE (2013)Google Scholar
  6. 6.
    Kuo, C.-H., Huang, C., Nevatia, R.: Inter-camera association of multi-target tracks by on-line learned appearance affinity models. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 383–396. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Multi-camera Object Tracking Challenge: ECCV Workshop on Visual Surveillance and Re-Identification (2014). http://mct.idealtest.org
  8. 8.
    Fleuret, F., Berclaz, J., Lengagne, R., Fua, P.: Multi-camera people tracking with a probabilistic occupancy map. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 267–282 (2008)CrossRefGoogle Scholar
  9. 9.
    Berclaz, J., Fleuret, F., Türetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1806–1819 (2011)CrossRefGoogle Scholar
  10. 10.
    Ferryman, J., Shahrokni, A.: An overview of the PETS 2009 challenge (2009)Google Scholar
  11. 11.
    D’Orazio, T., Leo, M., Mosca, N., Spagnolo, P., Mazzeo, P.L.: A semi-automatic system for ground truth generation of soccer video sequences. In: Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, pp. 559–564. IEEE (2009)Google Scholar
  12. 12.
    De Vleeschouwer, C., Chen, F., Delannay, D., Parisot, C., Chaudy, C., Martrou, E., Cavallaro, A., et al.: Distributed video acquisition and annotation for sport-event summarization. In: NEM summit 2008: Towards Future Media Internet (2008)Google Scholar
  13. 13.
    Zhang, S., Staudt, E., Faltemier, T., Roy-Chowdhury, A.: A camera network tracking (CamNeT) dataset and performance baseline. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 365–372, January 2015Google Scholar
  14. 14.
    Per, J., Kenk, V.S., Mandeljc, R., Kristan, M., Kovačič, S.: Dana36: a multi-camera image dataset for object identification in surveillance scenarios. In: 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 64–69. IEEE (2012)Google Scholar
  15. 15.
    Benenson, R., Omran, M., Hosang, J., Schiele, B.: Ten years of pedestrian detection, what have we learned? In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014 Workshops. LNCS, vol. 8926, pp. 613–627. Springer, Heidelberg (2015)Google Scholar
  16. 16.
    Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv: 1504.01942, April 2015
  17. 17.
    Bredereck, M., Jiang, X., Korner, M., Denzler, J.: Data association for multi-object Tracking-by-Detection in multi-camera networks. In: 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC), pp. 1–6, October 2012Google Scholar
  18. 18.
    Cai, Y., Medioni, G.: Exploring context information for inter-camera multiple target tracking. In: 2014 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 761–768, March 2014Google Scholar
  19. 19.
    Chen, K.W., Lai, C.C., Lee, P.J., Chen, C.S., Hung, Y.P.: Adaptive learning for target tracking and true linking discovering across multiple non-overlapping cameras. IEEE Trans. Multimedia 13(4), 625–638 (2011)CrossRefGoogle Scholar
  20. 20.
    Chen, X., An, L., Bhanu, B.: Multitarget tracking in nonoverlapping cameras using a reference set. IEEE Sens. J. 15(5), 2692–2704 (2015)CrossRefGoogle Scholar
  21. 21.
    Chen, X., Huang, K., Tan, T.: Direction-based stochastic matching for pedestrian recognition in non-overlapping cameras. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 2065–2068, September 2011Google Scholar
  22. 22.
    Daliyot, S., Netanyahu, N.S.: A framework for inter-camera association of multi-target trajectories by invariant target models. In: Park, J.-I., Kim, J. (eds.) ACCV Workshops 2012, Part II. LNCS, vol. 7729, pp. 372–386. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  23. 23.
    Das, A., Chakraborty, A., Roy-Chowdhury, A.K.: Consistent re-identification in a camera network. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 330–345. Springer, Heidelberg (2014)Google Scholar
  24. 24.
    Gilbert, A., Bowden, R.: Tracking objects across cameras by incrementally learning inter-camera colour calibration and patterns of activity. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 125–136. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  25. 25.
    Javed, O., Shafique, K., Rasheed, Z., Shah, M.: Modeling inter-camera space time and appearance relationships for tracking across non-overlapping views. Comput. Vis. Image Underst. 109(2), 146–162 (2008)CrossRefGoogle Scholar
  26. 26.
    Makris, D., Ellis, T., Black, J.: Bridging the gaps between cameras. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, June 2004Google Scholar
  27. 27.
    Jiuqing, W., Li, L.: Distributed optimization for global data association in non-overlapping camera networks. In: 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC), pp. 1–7, October 2013Google Scholar
  28. 28.
    Calderara, S., Cucchiara, R., Prati, A.: Bayesian-competitive consistent labeling for people surveillance. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 354–360 (2008)CrossRefGoogle Scholar
  29. 29.
    Zhang, S., Zhu, Y., Roy-Chowdhury, A.: Tracking multiple interacting targets in a camera network. Comput. Vis. Image Underst. 134, 64–73 (2015)CrossRefGoogle Scholar
  30. 30.
    Ayazoglu, M., Li, B., Dicle, C., Sznaier, M., Camps, O.: Dynamic subspace-based coordinated multicamera tracking. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2462–2469, November 2011Google Scholar
  31. 31.
    Kamal, A., Farrell, J., Roy-Chowdhury, A.: Information consensus for distributed multi-target tracking. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2403–2410, June 2013Google Scholar
  32. 32.
    Hamid, R., Kumar, R., Grundmann, M., Kim, K., Essa, I., Hodgins, J.: Player localization using multiple static cameras for sports visualization. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 731–738, June 2010Google Scholar
  33. 33.
    Martinel, N., Micheloni, C., Foresti, G.L.: Saliency weighted features for person re-identification. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8927, pp. 191–208. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-16199-0_14 Google Scholar
  34. 34.
    Zhao, R., Ouyang, W., Wang, X.: Unsupervised salience learning for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  35. 35.
    Bedagkar-Gala, A., Shah, S.: Multiple person re-identification using part based spatio-temporal color appearance model. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1721–1728, November 2011Google Scholar
  36. 36.
    Bedagkar-Gala, A., Shah, S.K.: Part-based spatio-temporal model for multi-person re-identification. Pattern Recogn. Lett. 33(14), 1908–1915 (2012). Novel Pattern Recognition-based Methods for Re-identification in Biometric ContextCrossRefGoogle Scholar
  37. 37.
    Cheng, D., Cristani, M., Stoppa, M., Bazzani, L., Murino, V.: Custom pictorial structures for re-identification. In: Proceedings of the British Machine Vision Conference, pp. 68.1–68.11. BMVA Press (2011). doi: 10.5244/C.25.68
  38. 38.
    Baltieri, D., Vezzani, R., Cucchiara, R.: Learning articulated body models for people re-identification. In: Proceedings of the 21st ACM International Conference on Multimedia, MM 2013, pp. 557–560. ACM, New York (2013)Google Scholar
  39. 39.
    Cheng, D., Cristani, M.: Person re-identification by articulated appearance matching. In: Gong, S., Cristani, M., Yan, S., Loy, C.C. (eds.) Person Re-Identification. Advances in Computer Vision and Pattern Recognition, pp. 139–160. Springer, London (2014)CrossRefGoogle Scholar
  40. 40.
    Baltieri, D., Vezzani, R., Cucchiara, R.: Mapping appearance descriptors on 3d body models for people re-identification. Int. J. Comput. Vis. 111(3), 345–364 (2015)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Brendel, W., Amer, M., Todorovic, S.: Multiobject tracking as maximum weight independent set. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1273–1280. IEEE (2011)Google Scholar
  42. 42.
    Shu, G., Dehghan, A., Oreifej, O., Hand, E., Shah, M.: Part-based multiple-person tracking with partial occlusion handling. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1815–1821. IEEE (2012)Google Scholar
  43. 43.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75(2), 247–266 (2007)CrossRefGoogle Scholar
  44. 44.
    Izadinia, H., Saleemi, I., Li, W., Shah, M.: (MP)\(^{2}\)T: multiple people multiple parts tracker. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 100–114. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  45. 45.
    Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1201–1208. IEEE (2011)Google Scholar
  46. 46.
    Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)Google Scholar
  47. 47.
    Butt, A.A., Collins, R.T.: Multiple target tracking using frame triplets. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part III. LNCS, vol. 7726, pp. 163–176. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  48. 48.
    Chari, V., Lacoste-Julien, S., Laptev, I., Sivic, J.: On pairwise costs for network flow multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5537–5545 (2015)Google Scholar
  49. 49.
    Collins, R.T.: Multitarget data association with higher-order motion models. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1744–1751. IEEE (2012)Google Scholar
  50. 50.
    Dehghan, A., Assari, S.M., Shah, M.: Gmmcp tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: CVPR, vol. 1, p. 2 (2015)Google Scholar
  51. 51.
    Kumar, R., Charpiat, G., Thonnat, M.: Multiple object tracking by efficient graph partitioning. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 445–460. Springer, Heidelberg (2015)Google Scholar
  52. 52.
    Shafique, K., Shah, M.: A noniterative greedy algorithm for multiframe point correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 51–65 (2005)CrossRefGoogle Scholar
  53. 53.
    Tang, S., Andres, B., Andriluka, M., Schiele, B.: Subgraph decomposition for multi-target tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5033–5041 (2015)Google Scholar
  54. 54.
    Wen, L., Li, W., Yan, J., Lei, Z., Yi, D., Li, S.Z.: Multiple target tracking based on undirected hierarchical relation hypergraph. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1282–1289. IEEE (2014)Google Scholar
  55. 55.
    Roshan Zamir, A., Dehghan, A., Shah, M.: GMCP-tracker: global multi-object tracking using generalized minimum clique graphs. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 343–356. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  56. 56.
    Yu, S.I., Meng, D., Zuo, W., Hauptmann, A.: The solution path algorithm for identity-aware multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3871–3879 (2016)Google Scholar
  57. 57.
    Yao, J., Odobez, J.M.: Multi-layer background subtraction based on color and texture. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)Google Scholar
  58. 58.
    Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  59. 59.
    Bansal, N., Blum, A., Chawla, S.: Correlation clustering. In: Foundations of Computer Science (2002)Google Scholar
  60. 60.
    Tan, J.: A note on the inapproximability of correlation clustering (2008)Google Scholar
  61. 61.
    Liu, C., Gong, S., Loy, C.C., Lin, X.: Person re-identification: what features are important? In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7583, pp. 391–401. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33863-2_39 CrossRefGoogle Scholar
  62. 62.
    Chen, W., Cao, L., Chen, X., Huang, K.: A novel solution for multi-camera object tracking. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 2329–2333. IEEE (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Ergys Ristani
    • 1
    Email author
  • Francesco Solera
    • 2
  • Roger Zou
    • 1
  • Rita Cucchiara
    • 2
  • Carlo Tomasi
    • 1
  1. 1.Computer Science DepartmentDuke UniversityDurhamUSA
  2. 2.Department of EngineeringUniversity of Modena and Reggio EmiliaModenaItaly

Personalised recommendations