MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization

  • Jianming Zhang
  • Shugao Ma
  • Stan Sclaroff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8694)


We propose a multi-expert restoration scheme to address the model drift problem in online tracking. In the proposed scheme, a tracker and its historical snapshots constitute an expert ensemble, where the best expert is selected to restore the current tracker when needed based on a minimum entropy criterion, so as to correct undesirable model updates. The base tracker in our formulation exploits an online SVM on a budget algorithm and an explicit feature mapping method for efficient model update and inference. In experiments, our tracking method achieves substantially better overall performance than 32 trackers on a benchmark dataset of 50 video sequences under various evaluation settings. In addition, in experiments with a newly collected dataset of challenging sequences, we show that the proposed multi-expert restoration scheme significantly improves the robustness of our base tracker, especially in scenarios with frequent occlusions and repetitive appearance variations.


Image Patch Entropy Minimization Robust Tracking Multiple Instance Learning Multiple Expert 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-319-10599-4_13_MOESM1_ESM.pdf (2.2 mb)
Electronic Supplementary Material (PDF 2,209 KB)


  1. 1.
    Avidan, S.: Support vector tracking. PAMI 26(8), 1064–1072 (2004)CrossRefGoogle Scholar
  2. 2.
    Avidan, S.: Ensemble tracking. PAMI 29(2), 261–271 (2007)CrossRefGoogle Scholar
  3. 3.
    Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. PAMI 33(8), 1619–1632 (2011)CrossRefGoogle Scholar
  4. 4.
    Bai, Q., Wu, Z., Sclaroff, S., Betke, M., Monnier, C.: Randomized ensemble tracking. In: ICCV (2013)Google Scholar
  5. 5.
    Chu, D.M., Smeulders, A.W.: Thirteen hard cases in visual tracking. In: AVSS (2010)Google Scholar
  6. 6.
    Dinh, T.B., Vo, N., Medioni, G.: Context tracker: Exploring supporters and distracters in unconstrained environments. In: CVPR (2011)Google Scholar
  7. 7.
    Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. In: BMVC (2006)Google Scholar
  8. 8.
    Grabner, H., Leistner, C., Bischof, H.: Semi-supervised on-line boosting for robust tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 234–247. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Grabner, M., Grabner, H., Bischof, H.: Learning features for tracking. In: CVPR (2007)Google Scholar
  10. 10.
    Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. In: NIPS (2005)Google Scholar
  11. 11.
    Hare, S., Saffari, A., Torr, P.H.: Struck: Structured output tracking with kernels. In: ICCV (2011)Google Scholar
  12. 12.
    He, S., Yang, Q., Lau, R.W., Wang, J., Yang, M.H.: Visual tracking via locality sensitive histograms. In: CVPR (2013)Google Scholar
  13. 13.
    Jia, X., Lu, H., Yang, M.H.: Visual tracking via adaptive structural local sparse appearance model. In: CVPR (2012)Google Scholar
  14. 14.
    Kalal, Z., Matas, J., Mikolajczyk, K.: Pn learning: Bootstrapping binary classifiers by structural constraints. In: CVPR (2010)Google Scholar
  15. 15.
    Kim, M., Kumar, S., Pavlovic, V., Rowley, H.: Face tracking and recognition with visual constraints in real-world videos. In: CVPR (2008)Google Scholar
  16. 16.
    Kwon, J., Lee, K.M.: Visual tracking decomposition. In: CVPR (2010)Google Scholar
  17. 17.
    Kwon, J., Lee, K.M.: Tracking by sampling trackers. In: ICCV (2011)Google Scholar
  18. 18.
    Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., van den Hengel, A.: A survey of appearance models in visual object tracking. arXiv preprint arXiv:1303.4803 (2013)Google Scholar
  19. 19.
    Maji, S., Berg, A.C.: Max-margin additive classifiers for detection. In: CVPR (2009)Google Scholar
  20. 20.
    Matthews, L., Ishikawa, T., Baker, S.: The template update problem. PAMI 26(6), 810–815 (2004)CrossRefGoogle Scholar
  21. 21.
    Mei, X., Ling, H.: Robust visual tracking and vehicle classification via sparse representation. PAMI 33(11), 2259–2272 (2011)CrossRefGoogle Scholar
  22. 22.
    Mei, X., Ling, H., Wu, Y., Blasch, E., Bai, L.: Minimum error bounded efficient l1 tracker with occlusion detection. In: CVPR (2011)Google Scholar
  23. 23.
    Supancic III, J.S., Ramanan, D.: Self-paced learning for long-term tracking. In: CVPR (2013)Google Scholar
  24. 24.
    Tang, F., Brennan, S., Zhao, Q., Tao, H.: Co-tracking using semi-supervised support vector machines. In: ICCV (2007)Google Scholar
  25. 25.
    Wang, D., Lu, H., Yang, M.H.: Least soft-thresold squares tracking. In: CVPR (2013)Google Scholar
  26. 26.
    Wang, Z., Vucetic, S.: Online training on a budget of support vector machines using twin prototypes. Statistical Analysis and Data Mining 3(3), 149–169 (2010)MathSciNetGoogle Scholar
  27. 27.
    Wu, Y., Lim, J., Yang, M.H.: Online object tracking: A benchmark. In: CVPR (2013)Google Scholar
  28. 28.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Computing Surveys (CSUR) 38(4), 13 (2006)CrossRefGoogle Scholar
  29. 29.
    Yu, Q., Dinh, T.B., Medioni, G.: Online tracking and reacquisition using co-trained generative and discriminative trackers. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 678–691. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  30. 30.
    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, Springer, Heidelberg (1994)Google Scholar
  31. 31.
    Zhang, T., Ghanem, B., Liu, S., Ahuja, N.: Low-rank sparse learning for robust visual tracking. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 470–484. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  32. 32.
    Zhong, W., Lu, H., Yang, M.H.: Robust object tracking via sparsity-based collaborative model. In: CVPR (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Jianming Zhang
    • 1
  • Shugao Ma
    • 1
  • Stan Sclaroff
    • 1
  1. 1.Department of Computer ScienceBoston UniversityUSA

Personalised recommendations