International Journal of Computer Vision

, Volume 111, Issue 2, pp 213–228 | Cite as

Locally Orderless Tracking

  • Shaul OronEmail author
  • Aharon Bar-Hillel
  • Dan Levi
  • Shai Avidan


Locally Orderless Tracking (LOT) is a visual tracking algorithm that automatically estimates the amount of local (dis)order in the target. This lets the tracker specialize in both rigid and deformable objects on-line and with no prior assumptions. We provide a probabilistic model of the target variations over time. We then rigorously show that this model is a special case of the Earth Mover’s Distance optimization problem where the ground distance is governed by some underlying noise model. This noise model has several parameters that control the cost of moving pixels and changing their color. We develop two such noise models and demonstrate how their parameters can be estimated on-line during tracking to account for the amount of local (dis)order in the target. We also discuss the significance of this on-line parameter update and demonstrate its contribution to the performance. Finally we show LOT’s tracking capabilities on challenging video sequences, both commonly used and new, displaying performance comparable to state-of-the-art methods.


Tracking EMD Noise model Online parameter update 


  1. Alterman, M., Schechner, Y., Perona, P., & Shamir, J. (2012). Independent components in dynamic refraction. CCIT Report 805.Google Scholar
  2. Avidan, S. (2005). Ensemble tracking. In CVPR, pp. 494–501.Google Scholar
  3. Babenko, B., Yang, M. H., & Belongie, S. (2009). Visual tracking with online multiple instance learning. In CVPR.Google Scholar
  4. Boltz, S., Nielsen, F., & Soatto, S. (2010). Earth mover distance on superpixels. In ICIP.Google Scholar
  5. Comaniciu, D. (2002). Bayesian kernel tracking. In DAGM-Symposium, pp. 438–445.Google Scholar
  6. Doucet, A., de Freitas, N., & Gordon, N. (2001). Sequential Monte Carlo methods in practice. Berlin: Springer.CrossRefzbMATHGoogle Scholar
  7. Elgammal, A., Duraiswami, R., Davis, L. S. (2003). Probabilistic tracking in joint feature-spatial spaces. In CVPR.Google Scholar
  8. Everingham, M., Van Gool, L. J., Williams, C. K. J., Winn, J. M., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88, 303.Google Scholar
  9. Ginneken, B. V., & Haar Romeny, B. M. T. (1999). Applications of locally orderless images. Scale-space theories in computer vision. Berlin: Springer.Google Scholar
  10. Godec, M., Roth, P. M., & Bischof, H. (2011). Hough-based tracking of non-rigid objects. In ICCV.Google Scholar
  11. Grabner, H., Grabner, M., & Bischof, H. (2006). Real-time tracking via online boosting. In BMVC.Google Scholar
  12. Hager, G. D., & Belhumeur, P. N. (1998). Efficient region tracking with parametric models of geometry and illumination. Pattern Analysis and Machine Intelligence, 20, 1025.CrossRefGoogle Scholar
  13. He, X., Zemel, R., & Ray, D. (2006). Learning and incorporating top-down cues in image segmentation. In ECCV.Google Scholar
  14. Heller, & Tompkins, C. B. G. (1956). An extension of a theorem of Dantzig’s. Linear Inequalities and Related Systems, vol. 38, pp. 247–254. Princeton, NJ: Annals of Mathematics Studies.Google Scholar
  15. Hoiem, D., Efros, A., & Hebert, M. (2005) Geometric context from a single image. In ICCV.Google Scholar
  16. Isard, M., & Blake, A. (1998). Condensation—conditional density propagation for visual tracking. International Journal of Computer Vision, 1, 5.Google Scholar
  17. Jia, X., Lu, H., & Yang, M. H. (2012). Visual tracking via adaptive structural local sparse appearance model. In CVPR.Google Scholar
  18. Kalal, Z., Mikolajczyk, K., & Matas, J. (2010). Tracking–learning–detection. TPAMI, 34, 1409.Google Scholar
  19. Koenderink, J. J., & Van Doorn, A. J. (1999). The structure of locally orderless images. International Journal of Computer Vision, 31, 159.Google Scholar
  20. Kwon, J., & Lee, K.M. (2010). Visual tracking decomposition. In CVPR.Google Scholar
  21. Levina, E., & Bickel, P. (2001). The earth mover’s distance Is the mallows distance: some insights from statistics. In ICCV. Google Scholar
  22. Levinshtein, A., Stere, A., Kutulakos, K. N., Fleet, D. J., Dickinson, S. J., & Siddiqi, K. (2009). Turbopixels: Fast superpixels using geometric flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 2290.Google Scholar
  23. Mei, X., Ling, H., Wu, Y., Blasch, E., & Bai, L. (2011) Minimum error bounded efficient L1 tracker with occlusion detection. In ICCV.Google Scholar
  24. Oron, S., Bar-Hillel, A., Levi, D., & Avidan, S. (2012). Locally Orderless Tracking. In CVPR.Google Scholar
  25. Peleg, S., Werman, M., & Rom, H. (1989). A unified approach to the change of resolution: Space and gray-level. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 739.CrossRefGoogle Scholar
  26. Ren, X., & Malik, J. (2003). Learning a classification model for segmentation. In ICCV.Google Scholar
  27. Ross, D., Lim, J., & Yang, M. H. (2004). Adaptive probabilistic visual tracking with incremental subspace update. In ECCV.Google Scholar
  28. Ross, D., Lim, J., Lin, R. S., & Yang, M. H. (2007). Incremental learning for robust visual tracking. International Journal of Computer Vision, 77, 125.CrossRefGoogle Scholar
  29. Rubner, Y., Tomasi, C., & Guibas, L. J. (2000). The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 40, 99.CrossRefzbMATHGoogle Scholar
  30. Santner, J., Leistner, C., Saffari, A., Pock, T., & Bischof, H. (2010). Prost: Parallel robust online simple tracking. In CVPR.Google Scholar
  31. Wang, S., Lu, H., Yang, F., & Yang, M. H. (2011). Superpixel tracking. In ICCV.Google Scholar
  32. Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A survey. ACM Computing Surveys (CSUR), 38, 13.CrossRefGoogle Scholar
  33. Zhao, Q., Yang, Z., & Tao, H. (2010). Differential earth mover’s distance with its applications to visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 274.CrossRefGoogle Scholar
  34. Zhong, W., Lu, H., & Yang, M. H. (2012). Robust object tracking via sparsity-based collaborative model. In CVPR.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Shaul Oron
    • 1
    Email author
  • Aharon Bar-Hillel
    • 2
  • Dan Levi
    • 3
  • Shai Avidan
    • 1
  1. 1.Tel Aviv UniversityTel AvivIsrael
  2. 2.Microsoft Research, Advanced Technology Labs Israel Microsoft - Haifa R&D CenterHaifa Israel
  3. 3.General Motors Advanced Technical CenterHerzliyaIsrael

Personalised recommendations