Learning for Multi-view 3D Tracking in the Context of Particle Filters

  • Juergen Gall
  • Bodo Rosenhahn
  • Thomas Brox
  • Hans-Peter Seidel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4292)


In this paper we present an approach to use prior knowledge in the particle filter framework for 3D tracking, i.e. estimating the state parameters such as joint angles of a 3D object. The probability of the object’s states, including correlations between the state parameters, is learned a priori from training samples. We introduce a framework that integrates this knowledge into the family of particle filters and particularly into the annealed particle filter scheme. Furthermore, we show that the annealed particle filter also works with a variational model for level set based image segmentation that does not rely on background subtraction and, hence, does not depend on a static background. In our experiments, we use a four camera set-up for tracking the lower part of a human body by a kinematic model with 18 degrees of freedom. We demonstrate the increased accuracy due to the prior knowledge and the robustness of our approach to image distortions. Finally, we compare the results of our multi-view tracking system quantitatively to the outcome of an industrial marker based tracking system.


Root Mean Square Error Joint Angle Particle Filter Weighted Euclidean Distance Base Image Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Doucet, A., de Freitas, N., Gordon, N. (eds.): Sequential Monte Carlo Methods in Practice. In: Statistics for Engineering and Information Science. Springer, New York (2001)Google Scholar
  2. 2.
    Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. Int. J. of Computer Vision 29, 5–28 (1998)CrossRefGoogle Scholar
  3. 3.
    Deutscher, J., Reid, I.: Articulated body motion capture by stochastic search. Int. J. of Computer Vision 61, 185–205 (2005)CrossRefGoogle Scholar
  4. 4.
    Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3D human figures using 2D image motion. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 702–718. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  5. 5.
    Sidenbladh, H., Black, M.J., Sigal, L.: Implicit probabilistic models of human motion for synthesis and tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 784–800. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Sminchisescu, C., Triggs, B.: Estimating articulated human motion with covariance scaled sampling. Int. J. of Robotics Research 22, 371–391 (2003)CrossRefGoogle Scholar
  7. 7.
    Sminchisescu, C., Jepson, A.: Generative modeling for continuous non-linearily embedded visual inference. In: Int. Conf. on Machine Learning (2004)Google Scholar
  8. 8.
    Brox, T., Rosenhahn, B., Kersting, U.G., Cremers, D.: Nonparametric density estimation for human pose tracking. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 546–555. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Brox, T., Rousson, M., Deriche, R., Weickert, J.: Unsupervised segmentation incorporating colour, texture, and motion. In: Petkov, N., Westenberg, M.A. (eds.) CAIP 2003. LNCS, vol. 2756, pp. 353–360. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. 10.
    Rosenhahn, B., Brox, T., Smith, D., Gurney, J., Klette, R.: A system for marker-less human motion estimation. Künstliche Intelligenz 1, 45–51 (2006)Google Scholar
  11. 11.
    Crisan, D., Doucet, A.: A survey of convergence results on particle filtering methods for practitioners. IEEE Transaction on Signal Processing 50, 736–746 (2002)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Kirkpatrick, S., Gelatt, C., Vecchi, M.: Optimization by simulated annealing. Science 220, 671–680 (1983)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Brox, T., Rosenhahn, B., Weickert, J.: Three-dimensional shape knowledge for joint image segmentation and pose estimation. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 109–116. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Parzen, E.: On estimation of a probability density function and mode. Annals of Mathematical Statistics 33, 1065–1076 (1962)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Silverman, B.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, London (1986)MATHGoogle Scholar
  16. 16.
    Mukundan, R., Ramakrishnan, K.: Moment Functions in Image Analysis: Theory and Application. World Scientific Publishing, Singapore (1998)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Juergen Gall
    • 1
  • Bodo Rosenhahn
    • 1
  • Thomas Brox
    • 2
  • Hans-Peter Seidel
    • 1
  1. 1.Max-Planck Institute for Computer ScienceSaarbrückenGermany
  2. 2.CVPR Group, Department of Computer ScienceUniversity of BonnBonnGermany

Personalised recommendations