Journal of Mathematical Imaging and Vision

, Volume 46, Issue 1, pp 103–120 | Cite as

Unscented Kalman Filtering on Riemannian Manifolds

  • Søren HaubergEmail author
  • François Lauze
  • Kim Steenstrup Pedersen


In recent years there has been a growing interest in problems, where either the observed data or hidden state variables are confined to a known Riemannian manifold. In sequential data analysis this interest has also been growing, but rather crude algorithms have been applied: either Monte Carlo filters or brute-force discretisations. These approaches scale poorly and clearly show a missing gap: no generic analogues to Kalman filters are currently available in non-Euclidean domains. In this paper, we remedy this issue by first generalising the unscented transform and then the unscented Kalman filter to Riemannian manifolds. As the Kalman filter can be viewed as an optimisation algorithm akin to the Gauss-Newton method, our algorithm also provides a general-purpose optimisation framework on manifolds. We illustrate the suggested method on synthetic data to study robustness and convergence, on a region tracking problem using covariance features, an articulated tracking problem, a mean value optimisation and a pose optimisation problem.


Riemannian manifolds Unscented Kalman filter Filtering theory Optimisation on manifolds 



The authors would like to thank the anonymous reviewers for detailed comments, which substantially improved the quality of the manuscript. Furthermore, Søren Hauberg would like to thank the Villum Foundation for financial support.

Supplementary material

(MP4 1.4 MB)

10851_2012_372_MOESM3_ESM.mp4 (13.3 mb)
(MP4 13.3 MB)

(MP4 4.6 MB)

(MP4 4.7 MB)

(MP4 899 kB)

(MP4 15.0 MB)

(MP4 4.8 MB)

(MP4 4.7 MB)


  1. 1.
    Balan, A.O., Sigal, L., Black, M.J.: A quantitative evaluation of video-based 3D person tracking. In: 2nd Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 349–356 (2005) CrossRefGoogle Scholar
  2. 2.
    Bandouch, J., Engstler, F., Beetz, M.: Accurate human motion capture using an ergonomics-based anthropometric human model. In: AMDO’08: Proceedings of the 5th International Conference on Articulated Motion and Deformable Objects. Lecture Notes in Computer Science, vol. 5098, pp. 248–258. Springer, Berlin (2008) CrossRefGoogle Scholar
  3. 3.
    Bell, B.M., Cathey, F.W.: The iterated Kalman filter update as a Gauss-Newton method. IEEE Trans. Autom. Control 38, 294–297 (1993) MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Cappé, O., Godsill, S.J., Moulines, E.: An overview of existing methods and recent advances in sequential Monte Carlo. Proc. IEEE 95(5), 899–924 (2007) CrossRefGoogle Scholar
  5. 5.
    do Carmo, M.P.: Riemannian Geometry. Birkhäuser, Boston (1992) zbMATHCrossRefGoogle Scholar
  6. 6.
    Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis. 22, 61–79 (1997) zbMATHCrossRefGoogle Scholar
  7. 7.
    Engell-Nørregård, M., Erleben, K.: A projected back-tracking line-search for constrained interactive inverse kinematics. Comput. Graph. 35(2), 288–298 (2011) CrossRefGoogle Scholar
  8. 8.
    Erleben, K., Sporring, J., Henriksen, K., Dohlmann, H.: Physics Based Animation. Charles River Media, Newton Center (2005) Google Scholar
  9. 9.
    Fletcher, P.T., Joshi, S.: Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Process. 87, 250–262 (2007) zbMATHCrossRefGoogle Scholar
  10. 10.
    Fletcher, P.T., Lu, C., Pizer, S.M., Joshi, S.: Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Trans. Med. Imaging 23(8), 995–1005 (2004) CrossRefGoogle Scholar
  11. 11.
    Hairer, E., Lubich, C., Wanner, G.: Geometric Numerical Integration: Structure Preserving Algorithms for Ordinary Differential Equations. Springer, Berlin (2004) Google Scholar
  12. 12.
    Hauberg, S., Pedersen, K.S.: Stick it! Articulated tracking using spatial rigid object priors. In: Asian Conference on Computer Vision. Lecture Notes in Computer Science, vol. 6494. Springer, Berlin (2010) Google Scholar
  13. 13.
    Hauberg, S., Pedersen, K.S.: Predicting articulated human motion from spatial processes. Int. J. Comput. Vis. 94, 317–334 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Hauberg, S., Pedersen, K.S.: HUMIM software for articulated tracking. Tech. Rep. 01/2012, Department of Computer Science, University of Copenhagen (2012) Google Scholar
  15. 15.
    Hauberg, S., Sommer, S., Pedersen, K.S.: Gaussian-like spatial priors for articulated tracking. In: ECCV. Lecture Notes in Computer Science, vol. 6311, pp. 425–437. Springer, Berlin (2010) Google Scholar
  16. 16.
    Hauberg, S., Sommer, S., Pedersen, K.S.: Natural metrics and least-committed priors for articulated tracking. Image Vis. Comput. 30(6–7), 453–461 (2012) CrossRefGoogle Scholar
  17. 17.
    Julier, S.J., Uhlmann, J.K.: A new extension of the Kalman filter to nonlinear systems. In: International Symposium Aerospace/Defense Sensing, Simulation and Controls, pp. 182–193 (1997) Google Scholar
  18. 18.
    Kalman, R.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(D), 35–45 (1960) CrossRefGoogle Scholar
  19. 19.
    Karcher, H.: Riemannian center of mass and mollifier smoothing. Commun. Pure Appl. Math. 30(5), 509–541 (1977) MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Kendall, D.G.: Shape manifolds, procrustean metrics, and complex projective spaces. Bull. Lond. Math. Soc. 16(2), 81–121 (1984) MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Kjellström, H., Kragić, D., Black, M.J.: Tracking people interacting with objects. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 747–754 (2010) Google Scholar
  22. 22.
    Kraft, E.: A quaternion-based unscented Kalman filter for orientation tracking. In: Proceedings of the Sixth International Conference on Information Fusion, pp. 47–54 (2003) Google Scholar
  23. 23.
    Kwon, J., Lee, K.M.: Monocular SLAM with locally planar landmarks via geometric rao-blackwellized particle filtering on Lie groups. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1522–1529 (2010) Google Scholar
  24. 24.
    Kwon, J., Lee, K.M., Park, F.C.: Visual tracking via geometric particle filtering on the affine group with optimal importance functions. In: Computer Vision and Pattern Recognition, pp. 991–998 (2009) Google Scholar
  25. 25.
    Lewis, F.L.: Optimal Estimation: With an Introduction to Stochastic Control Theory. Wiley, New York (1986) zbMATHGoogle Scholar
  26. 26.
    Li, R., Chellappa, R.: Aligning spatio-temporal signals on a special manifold. In: ECCV. Lecture Notes in Computer Science, vol. 6315, pp. 547–560. Springer, Berlin (2010) Google Scholar
  27. 27.
    Liu, X., Srivastava, A., Gallivan, K.: Optimal linear representations of images for object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 662–666 (2004) CrossRefGoogle Scholar
  28. 28.
    van der Merwe, R., Doucet, A., Freitas, N.D., Wan, E.: The unscented particle filter. In: Advances in Neural Information Processing Systems (NIPS 2000), vol. 13, pp. 584–590. MIT Press, Cambridge (2001) Google Scholar
  29. 29.
    Misner, C., Thorne, K., Wheeler, J.: Gravitation. W.H. Freeman, New York (1973) Google Scholar
  30. 30.
    Pennec, X.: Intrinsic statistics on Riemannian manifolds: basic tools for geometric measurements. J. Math. Imaging Vis. 25(1), 127–154 (2006) MathSciNetCrossRefGoogle Scholar
  31. 31.
    Pennec, X., Fillard, P., Ayache, N.: A Riemannian framework for tensor computing. Int. J. Comput. Vis. 66, 41–66 (2004) CrossRefGoogle Scholar
  32. 32.
    Poppe, R.: Vision-based human motion analysis: an overview. Comput. Vis. Image Underst. 108(1–2), 4–18 (2007) CrossRefGoogle Scholar
  33. 33.
    Porikli, F., Tuzel, O., Meer, P.: Covariance tracking using model update based on Lie algebra. In: Computer Vision and Pattern Recognition, vol. 1, pp. 728–735 (2006) Google Scholar
  34. 34.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989) CrossRefGoogle Scholar
  35. 35.
    Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3D human figures using 2D image motion. In: ECCV, vol. II. Lecture Notes in Computer Science, vol. 1843, pp. 702–718. Springer, Berlin (2000) Google Scholar
  36. 36.
    Sigal, L., Black, M.J.: HumanEva: synchronized video and motion capture dataset for evaluation of articulated human motion. Tech. Rep. CS-06-08, Brown University (2007) Google Scholar
  37. 37.
    Singhal, S., Wu, L.: Training multilayer perceptrons with the extended Kalman algorithm. In: Advances in Neural Information Processing Systems, vol. 1, pp. 133–140 (1989) Google Scholar
  38. 38.
    Sipos, B.J.: Application of the Manifold-Constrained unscented Kalman filter. In: Position, Location and Navigation Symposium, IEEE/ION, pp. 30–43 (2008) Google Scholar
  39. 39.
    Sommer, S., Tatu, A., Chen, C., Jørgensen, D.R., de Bruijne, M., Loog, M., Nielsen, M., Lauze, F.: Bicycle chain shape models. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 157–163. IEEE Computer Society, Los Alamitos (2009) CrossRefGoogle Scholar
  40. 40.
    Sommer, S., Lauze, F., Nielsen, M.: The differential of the exponential map, Jacobi fields and exact principal geodesic analysis. CoRR (2010). arXiv:1008.1902
  41. 41.
    Srivastava, A., Klassen, E.: Bayesian and geometric subspace tracking. Adv. Appl. Probab. 36(1), 43–56 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Subbarao, R., Meer, P.: Nonlinear mean shift over Riemannian manifolds. Int. J. Comput. Vis. 84(1), 1–20 (2009) CrossRefGoogle Scholar
  43. 43.
    Tidefelt, H., Schön, T.B.: Robust point-mass filters on manifolds. In: Proceedings of the 15th IFAC Symposium on System Identification (SYSID), pp. 540–545 (2009) Google Scholar
  44. 44.
    Tuzel, O., Porikli, F., Meer, P.: Region covariance: A fast descriptor for detection and classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV. Lecture Notes in Computer Science, vol. 3952, pp. 589–600. Springer, Berlin/Heidelberg (2006) Google Scholar
  45. 45.
    Tyagi, A., Davis, J.W.: A recursive filter for linear systems on Riemannian manifolds. In: Computer Vision and Pattern Recognition, pp. 1–8 (2008) Google Scholar
  46. 46.
    Wan, E.A., van der Merwe, R.: The unscented Kalman filter for nonlinear estimation. In: Adaptive Systems for Signal Processing, Communications, and Control Symposium, IEEE, pp. 153–158 (2002) Google Scholar
  47. 47.
    Ward, R.C.: Numerical computation of the matrix exponential with accuracy estimate. SIAM J. Numer. Anal. 14, 600–610 (1977) MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Wu, Y., Wu, B., Liu, J., Lu, H.: Probabilistic tracking on Riemannian manifolds. In: International Conference on Pattern Recognition, pp. 1–4 (2008) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Søren Hauberg
    • 1
    Email author
  • François Lauze
    • 2
  • Kim Steenstrup Pedersen
    • 2
  1. 1.Perceiving SystemsMax Planck Institute for Intelligent SystemsStuttgartGermany
  2. 2.Dept. of Computer ScienceUniversity of CopenhagenCopenhagenDenmark

Personalised recommendations