International Journal of Computer Vision

, Volume 104, Issue 2, pp 198–219 | Cite as

Multiframe Many–Many Point Correspondence for Vehicle Tracking in High Density Wide Area Aerial Videos

  • Imran SaleemiEmail author
  • Mubarak Shah


This paper presents a novel framework for tracking thousands of vehicles in high resolution, low frame rate, multiple camera aerial videos. The proposed algorithm avoids the pitfalls of global minimization of data association costs and instead maintains multiple object-centric associations for each track. Representation of object state in terms of many to many data associations per track is proposed and multiple novel constraints are introduced to make the association problem tractable while allowing sharing of detections among tracks. Weighted hypothetical measurements are introduced to better handle occlusions, mis-detections and split or merged detections. A two-frame differencing method is presented which performs simultaneous moving object detection in both. Two novel contextual constraints of vehicle following model, and discouragement of track intersection and merging are also proposed. Extensive experiments on challenging, ground truthed data sets are performed to show the feasibility and superiority of the proposed approach. Results of quantitative comparison with existing approaches are presented, and the efficacy of newly introduced constraints is experimentally established. The proposed algorithm performs better and faster than global, 1–1 data association methods.


Wide area aerial surveillance Multi target tracking Multiframe data association Multiple hypothesis Multiple candidate tracking CLIF dataset 

Supplementary material

11263_2013_624_MOESM1_ESM.txt (3 kb)
Supplementary material 1 (txt 3 KB)
11263_2013_624_MOESM2_ESM.avi (4.8 mb)
Supplementary material 2 (avi 4922 KB)
11263_2013_624_MOESM3_ESM.avi (4.8 mb)
Supplementary material 3 (avi 4868 KB)
11263_2013_624_MOESM4_ESM.avi (7 mb)
Supplementary material 4 (avi 7158 KB)
11263_2013_624_MOESM5_ESM.avi (2.8 mb)
Supplementary material 5 (avi 2895 KB)
11263_2013_624_MOESM6_ESM.avi (5.8 mb)
Supplementary material 6 (avi 5900 KB)


  1. Ablavsky, V., Thangali, A., & Sclaroff, S. (2008). Layered graphical models for tracking partially-occluded objects. In CVPR, Anchorage.Google Scholar
  2. Bazzani, L., Cristani, M., & Murino, V. (2010). Collaborative particle filters for group tracking. In ICIP, Hong Kong.Google Scholar
  3. Berclaz, J., Fleuret, F., Turetken, E., & Fua, P. (2011). Multiple object tracking using k-shortest paths optimization. IEEE Transactions on Pattern Analysis and Machine, 33(9), 1806–1819.CrossRefGoogle Scholar
  4. Breitenstein, M., Reichlin, F., Leibe, B., Koller-Meier, E., & Gool, L. V. (2009). Robust tracking-by-detection using a detector confidence particle filter. In ICCV, Kyoto.Google Scholar
  5. Cox, I., & Hingorani, S. (1996). An efficient implementation of Reid’s multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking. IEEE TPAMI, 18(2), 138–150.CrossRefGoogle Scholar
  6. Cox, I., Miller, M., Danchick, R., & Newnam, G. (1997). A comparison of two algorithms for determining ranked assignments with application to multitarget tracking and motion correspondence. IEEE Transaction of AES, 33(1), 295–301.Google Scholar
  7. Cucchiara, R., Piccardi, M., & Mello, P. (2000). Image analysis and rule-based reasoning for a traffic monitoring system. IEEE Transaction of ITS, 1(2), 119–130.Google Scholar
  8. Danchick, R., & Newnam, G. (2006). Reformulating Reid’s MHT method with generalised Murty k-best ranked linear assignment algorithm. IEE Proceedings Radar, Sonar and Navigation, 153(1), 22.CrossRefGoogle Scholar
  9. Durrant-Whyte, H. (1988). Sensor models and multisensor integration. International Journal of Robotics Research, 7(6), 97–113.CrossRefGoogle Scholar
  10. Edie, L. (1960). Car following and steady state theory for non-congested traffic. Operations Research, 9, 66–76.MathSciNetCrossRefGoogle Scholar
  11. Gazis, D., Herman, R., & Potts, R. (1959). Car following theory of steady state traffic flow. Operations Research, 7, 499–505.MathSciNetCrossRefGoogle Scholar
  12. Grabner, H., Leistner, C., & Bischof, H. (2008). Semi-supervised on-line boosting for robust tracking. In ECCV, Marseille.Google Scholar
  13. Hinton, G. (1999). Products of experts. In Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN 99) (pp. 1–6).Google Scholar
  14. Huang, K., Wang, L., Tan, T., & Maybank, S. (2008). A real-time object detecting and tracking system for outdoor night surveillance. Pattern Recognition, 41, 432–444.zbMATHCrossRefGoogle Scholar
  15. Kang, J., Cohen, I., & Medioni, G. (2003). Soccer player tracking across uncalibrated camera streams. In ICCV: PETS Workshop, Nice.Google Scholar
  16. Kikuchi, S., & Chakroborty, P. (1992). Car following model based on fuzzy inference system. Transportation Research Record, 1365, 82–91.Google Scholar
  17. Kuhn, H. (1955). The Hungarian method for solving the assignment problem. Naval Research Logistics Quarterly, 2, 83–97.MathSciNetCrossRefGoogle Scholar
  18. Mann, S., & Picard, R. (1997). Video orbits of projective group: An approach to featureless estimation of parameters. IEEE TIP, 6, 1281–1295.Google Scholar
  19. Munkres, J. (1957). Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics, 5(1), 32–38.MathSciNetzbMATHCrossRefGoogle Scholar
  20. Murty, K. (1968). An algorithm for ranking all the assignments in order of increasing cost. Operations Research, 16(3), 682–687.zbMATHCrossRefGoogle Scholar
  21. Nagel, K., & Schreckenberg, M. (1992). A cellular automaton model for freeway traffic. Journal of Physics I: France, 2(12), 2221–2229.CrossRefGoogle Scholar
  22. Newell, G. (1961). Nonlinear effects in the dynamics of car following. Operations Research, 9(2), 209–229.MathSciNetzbMATHCrossRefGoogle Scholar
  23. Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66.MathSciNetCrossRefGoogle Scholar
  24. Perera, A., Srinivas, C., Hoogs, A., Brooksby, G., & Hu, W. (2006). Multi-object tracking through simultaneous long occlusions and split–merge conditions. In CVPR, New York.Google Scholar
  25. Pirsiavash, H., Ramanan, D., & Fowlkes, C. (2011). Globally-optimal greedy algorithms for tracking a variable number of objects. In CVPR, Colorado Springs (pp. 1201–1208).Google Scholar
  26. Porikli, F., & Pan., P. (2009). Regressed importance sampling on manifolds for efficient object tracking. AVSS, Genoa.Google Scholar
  27. Reid, D. (1979). An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, 24, 843–854.CrossRefGoogle Scholar
  28. Reilly, V., Idrees, H., & Shah, M. (2010). Detection and tracking of large number of targets in wide area surveillance. In ECCV, Heraklion.Google Scholar
  29. Schubert, R., Richter, E., & Wanielik, G. (2008). Comparison and evaluation of advanced motion models for vehicle tracking. In ICIF, Shanghai. Google Scholar
  30. Schulz, D., Burgard, W., Fox, D., & Cremers, A. (2001). Tracking multiple moving targets with a mobile robot using particle filters and statistical data association. In ICRA, Seoul.Google Scholar
  31. Shafique, K., & Shah, M. (2005). A non-iterative greedy algorithm for multi-frame point correspondence. IEEE TPAMI, 27(1), 51–65.CrossRefGoogle Scholar
  32. Shalom, Y., & Fortmann, T. (1988). Tracking and data association. London: Academic Press.zbMATHGoogle Scholar
  33. Shamos, M., & Hoey, D. (1976). Geometric intersection problems. In SFCS, Houston (pp. 208–215).Google Scholar
  34. Song, B., Jeng, T., Staudt, E., & Roy-chowdhury, A. (2010). A stochastic graph evolution framework for robust multi-target tracking. In ECCV, Crete.Google Scholar
  35. Stauffer, C., & Grimson, W. (2000). Learning patterns of activity using real-time tracking. IEEE TPAMI, 22(8), 747–757.CrossRefGoogle Scholar
  36. USAF. (2006). Columbus large image Format dataset.
  37. Vezzani, R., Baltieri, D., & Cucchiara, R. (2009). Pathnodes integration of standalone particle filters for people tracking on distributed surveillance systems. In ICIAP, Vietri sul Mare.Google Scholar
  38. Wang, G., Xiao, D., & Gu, J. (2008). Review on vehicle detection based on video for traffic surveillance. In ICAL, Qindao (pp. 2961–2966).Google Scholar
  39. Xiao, J., Cheng, H., Han, F., & Sawhney, H. (2008). Geo-spatial aerial video processing for scene understanding and object tracking. In CVPR, Anchorage.Google Scholar
  40. Xiao, J., Cheng, H., Sawhney, H., & Han, F. (2010). Vehicle detection and tracking in wide field-of-view aerial video. In CVPR, San Francisco.Google Scholar
  41. Xing, J., Ai, H., & Lao, S. (2009). Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In CVPR, Miami.Google Scholar
  42. Yang, M., Lv, F., Xu, W., & Gong, Y. (2009). Detection driven adaptive multi-cue integration for multiple human tracking. In ICCV, Kyoto.Google Scholar
  43. Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A survey. ACM Computing Surveys, 38, 1–45.CrossRefGoogle Scholar
  44. Yin, Z., & Collins, R. (2006). Moving object localization in thermal imagery by forward–backward MHI. In OTCBVS, Kokomo.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Department of Electrical Engineering & Computer ScienceUniversity of Central FloridaOrlandoUSA

Personalised recommendations