Abstract.
Over the past few years researchers have been investigating the enhancement of visual tracking performance by devising trackers that simultaneously make use of several different features. In this paper we investigate the combination of synchronous visual trackers that use different features while treating the trackers as “black boxes”. That is, instead of fusing the usage of the different types of data as has been performed in previous work, the combination here is allowed to use only the trackers' output estimates, which may be modified before their propagation to the next time step. We propose a probabilistic framework for combining multiple synchronous trackers, where each separate tracker outputs a probability density function of the tracked state, sequentially for each image. The trackers may output either an explicit probability density function, or a sample-set of it via Condensation. Unlike previous tracker combinations, the proposed framework is fairly general and allows the combination of any set of trackers of this kind, even in different state-spaces of different dimensionality, under a few reasonable assumptions. The combination may consist of different trackers that track a common object, as well as trackers that track separate, albeit related objects, thus improving the tracking performance of each object. The benefits of merely using the final estimates of the separate trackers in the combination are twofold. Firstly, the framework for the combination is fairly general and may be easily used from the software aspects. Secondly, the combination may be performed in a distributed setting, where each separate tracker runs on a different site and uses different data, while avoiding the need to share the data. The suggested framework was successfully tested using various state-spaces and datasets, demonstrating that fusing the trackers' final distribution estimates may indeed be applicable.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baker, T. and Strens, M. 1998. Representation of uncertainty in spatial target tracking. In Proceedings of the 14th International Conference on Pattern Recognition, pp. 1339–1342.
Bar-Shalom, Y. and Fortmann, T. 1988. Tracking and Data Association. Academic Press.
Bar-Shalom, Y. ed. 1992. Multitarget-multisensor tracking. Artech House.
Beal, M.J., Jojic, N., and Attias, H. 2003. A graphical model for audiovisual object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(7):828–836.
Blake, A., Curwen, R., and Zisserman, A. 1993. A framework for spatio-temporal control in the tracking of visual contours. International Journal of Computer Vision, 11(2):127–145.
Blatt, D. and Hero, A. 2004. Distributed maximum likelihood estimation for sensor networks. In Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 929–932.
Collins, R. and Liu, Y. 2003. On-line selection of discriminative tracking features. In Proceedings of the 9th IEEE International Conference on Computer Vision, pp. 346–352.
Darrell, T., Gordon, G., Harville, M., and Woodfill, J. 2000. Integrated person tracking using stereo, color, and pattern detection. International Journal of Computer Vision, 37(2):175–185.
Gil, S., Milanese, R., and Pun, T. 1996. Combining multiple motion estimates for vehicle tracking. In Proceedings of the 4th European Conference on Computer Vision, vol. 2, pp. 307–320.
Hue, C., Le Cadre, J.P., and Pérez, P. 2001. A particle filter to track multiple objects. In Proceedings of the 2001 IEEE Workshop on Multi-Object Tracking.
Isard, M. and Blake, A. 1998. Condensation -- conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1):5–28.
Isard, M. and MacCormick, J. 2001. BraMBLe: A bayesian multiple-blob tracker. In Proceedings of the 8th IEEE International Conference on Computer Vision, pp. 34–41.
Isard, M. 2003. PAMPAS: Real-valued graphical models for computer vision. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 613–620.
Jepson, A.D., Fleet, D.J., and El-Maraghi, T. F. 2001. Robust online appearance models for visual tracking. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 415–422.
Leichter, I., Lindenbaum, M., and Rivlin. E. 2004a. A probabilistic cooperation between trackers of coupled objects. In Proceedings of the 2004 IEEE International Conference on Image Processing, vol. 2, pp. 1045–1048.
Leichter, I., Lindenbaum, M., and Rivlin. E. 2004b. A probabilsitic framework for combining tracking algorithms. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 445–451.
MacCormick, J. and Blake, A. 2000b. A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision, 39(1):57–71.
McCane, B., Galvin, B., and Novins, K. 2002. Algorithmic fusion for more rubust feature tracking. International Journal of Computer Vision, 49(1):79–89.
Okuma, K., Taleghani, A., de Freitas, N., Little, J.J., and Lowe, S.G. 2004. A boosted particle filter: Multitarget detection and tracking. In Proceedings of the 8th European Conference on Computer Vision, vol. 1, pp. 28–39.
Papoulis, A. 1991. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 3rd edition.
Pérez, P., Hue, C., Vermaak, J., and Gangnet, M. 2002. Color-based probabilistic tracking. In Proceedings of the 7th European Conference on Computer Vision, pp. 661–675.
Pérez, P., Vermaak, J., and Blake, A. 2004. Data fusion for visual tracking with particles. Proceedings of the IEEE, 92(3):495–513.
Rasmussen, C. and Hager, G.D. 2001. Probabilistic data association methods for tracking complex visual objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):560–576.
Rosenberg, Y., and Werman, M. 1997a. A general filter for measurements with any probability distribution. In Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 654–659.
Rosenberg, Y. and Werman, M. 1997b. Representing local motion as a probability distribution matrix applied to object tracking. In Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 106–111.
Scott, D.W. 1992. Multivariate Density Estimation. New York: Wiley.
Shearer, K., Wong, K.D., and Venkatesh, S. 2001. Combining multiple tracking algorithms for improved general performance. Pattern Recognition, 34(6):1257–1269.
Sidenbladh, H. and Wirkander, S.L. 2003. Tracking random sets of vehicles in terrain. In Proceedings of the 2003 IEEE Workshop on Multi-Object Tracking.
Siebel, N.T. and Maybank, S. 2002. Fusion of multiple tracking algorithms for robust people tracking. In Proceedings of the 7th European Conference on Computer Vision, vol. 4, pp. 373–387.
Spengler, M. and Schiele, B. 2003. Towards robust multi-cue integration for visual tracking. Machine Vision and Applications, 14:50–58.
Sudderth, E.B., Ihler, A.T., Freeman, W.T., and Willsky, A.S. 2003. Nonparametric belief propagation. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 605–612.
Toyama, K. and Blake, A. 2002. Probabilistic tracking with exemplars in a metric space. International Journal of Computer Vision, 48(1):9–19.
Toyama, A. and Hager, G.D. 1995. Tracker fusion for robustness in visual feature tracking. SPIE -- The International Society for Optical Engineering, 2569:38–49.
Triesch, J. and von der Malsburg, C. 2000. Self-organized integration of adaptive visual cues for face tracking. In Proceedings of the 4th International Conference on Automatic Face and Gesture Recognition.
Vermaak, J., Doucet, A., and Pérez, P. 2003. Maintaining multi-modality through mixture tracking. In Proceedings of the 9th IEEE International Conference on Computer Vision, vol. 2, pp. 1110–1116.
Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518.
Wu, Y. and Huang, T.S. 2004. Robust visual tracking by integrating multiple cues based on co-inference learning. International Journal of Computer Vision, 58(1):55–71.
Proceedings of the 2001 IEEE Workshop on Multi-Object Tracking, Proceedings of the 2003 IEEE Workshop on Multi-Object Tracking.
http://www.visualsurveillance.org and related links.
Proceedings of the IEEE International Workshop Series on Performance Evaluation of Tracking and Surveillance.
Author information
Authors and Affiliations
Corresponding author
Additional information
First online version published in October, 2005
Electronic supplementary material
Supplementary material (5.00 MB)
Supplementary material (5.00 MB)
Supplementary material (5.00 MB)
Supplementary material (5.00 MB)
Supplementary material (5.00 MB)
Supplementary material (5.00 MB)
Supplementary material (5.00 MB)
Rights and permissions
About this article
Cite this article
Leichter, I., LINDENBAUM, M. & RIVLIN, E. A General Framework for Combining Visual Trackers – The "Black Boxes" Approach. Int J Comput Vision 67, 343–363 (2006). https://doi.org/10.1007/s11263-006-5568-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-006-5568-2