Skip to main content

Rate-Invariant Analysis of Covariance Trajectories

Journal of Mathematical Imaging and Vision volume 60, pages 1306–1323 (2018)Cite this article

Abstract

Statistical analysis of dynamic systems, such as videos and dynamic functional connectivity, is often translated into a problem of analyzing trajectories of relevant features, particularly covariance matrices. As an example, in video-based action recognition, a natural mathematical representation of activity videos is as parameterized trajectories on the set of symmetric, positive-definite matrices (SPDMs). The execution rates of actions, implying arbitrary parameterizations of trajectories, complicate their analysis. To handle this challenge, we represent covariance trajectories using transported square-root vector fields, constructed by parallel translating scaled-velocity vectors of trajectories to their starting points. The space of such representations forms a vector bundle on the SPDM manifold. Using a natural Riemannian metric on this vector bundle, we approximate geodesic paths and geodesic distances between trajectories in the space of this vector bundle. This metric is invariant to the action of the re-parameterization group, and leads to a rate-invariant analysis of trajectories. In the process, we remove the parameterization variability and temporally register trajectories. We demonstrate this framework in multiple contexts, using both generative statistical models and discriminative data analysis. The latter is illustrated using several applications involving video-based action recognition and dynamic functional connectivity analysis.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

References

  1. Afsari, B., Tron, R., Vidal, R.: On the convergence of gradient descent for finding the Riemannian center of mass. SIAM J. Control Optim. 51, 2230–2260 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  2. Aron, A.R., Robbins, T.W., Poldrack, R.A.: Inhibition and the right inferior frontal cortex. Trends Cogn. Sci. (Regul. Ed.) 8(4), 170–177 (2004)

    Article  Google Scholar 

  3. Arsigny, V., Fillard, P., Pennec, X., Ayache, N.: Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magn. Reson. Med. 56(2), 411–421 (2006)

    Article  Google Scholar 

  4. Basser, P.J., Pierpaoli, C.: Microstructural and physiological features of tissues elucidated by quantitative-diffusion-tensor MRI. J. Magn. Reson. B 111(3), 209–219 (1996)

    Article  Google Scholar 

  5. Brigant, A.L.: Computing distances and geodesics between manifold-valued curves in the SRV framework. arXiv:1601.02358 (2016)

  6. Brigant, A.L., Arnaudon, M., Barbaresco, F.: Reparameterization invariant metric on the space of curves. arXiv:1507.06503 (2015)

  7. Celledoni, E., Eslitzbichler, M., Schmeding, A.: Shape analysis on lie groups with applications in computer animation. J. Geom. Mech. 8(3), 273–304 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dai, M., Zhang, Z., Srivastava, A.: Testing stationarity of brain functional connectivity using change-point detection in fMRI data. In: CVPR Workshops Diff-CVML, pp. 981–989 (2016)

  9. Dai, M., Zhang, Z., Srivastava, A.: Discovering change-point patterns in dynamic functional brain connectivity of a population. In: IPMI (2017)

  10. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on CVPR, vol. 2, pp. 886–893 (2005)

  11. Destrieux, C., Fischl, B., Dale, A., Halgren, E.: Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. Neuroimage 53(1), 1–15 (2010)

    Article  Google Scholar 

  12. Dryden, I.L., Koloydenko, A.A., Zhou, D.: Non-Euclidean statistics for covariance matrices with applications to diffusion tensor imaging. Ann. Appl. Stat. 3(3), 1102?1123 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  13. Faraki, M., Palhang, M., Sanderson, C.: Log-Euclidean bag of words for human action recognition. IET Comput. Vision 9(3), 331–339 (2014)

    Article  Google Scholar 

  14. Glasser, M.F., Sotiropoulos, S.N., Wilson, J.A., Coalson, T.S., Fischl, B., Andersson, J.L., Xu, J., Jbabdi, S., Webster, M., Polimeni, J.R., Van Essen, D.C., Jenkinson, M.: The minimal preprocessing pipelines for the human connectome project. Neuroimage 80, 105–124 (2013)

    Article  Google Scholar 

  15. Guo, K., Ishwar, P., Konrad, J.: Action recognition in video by sparse representation on covariance manifolds of silhouette tunnels. In: Proceedings of the 20th International Conference on Recognizing Patterns in Signals, Speech, Images, and Videos, pp. 294–305 (2010)

  16. Harandi, M., Hartley, R., Shen, C., Lovell, B., Sanderson, C.: Extrinsic methods for coding and dictionary learning on Grassmann manifolds. Int. J. Comput. Vision 114(2), 113–136 (2015)

    Article  MathSciNet  Google Scholar 

  17. Harandi, M.T., Sanderson, C., Wiliem, A., Lovell, B.C.: Kernel analysis over Riemannian manifolds for visual recognition of actions, pedestrians and textures. In: Proceedings of the 2012 IEEE Workshop on the Applications of Computer Vision, pp. 433–439 (2012)

  18. Hutchison, R.M., et al.: Dynamic functional connectivity: promise, issues, and interpretations. Neuroimage 80, 360–378 (2013)

    Article  Google Scholar 

  19. Jost, J.: Riemannian Geometry and Geometric Analysis. Springer, New York (2005)

    MATH  Google Scholar 

  20. Jupp, P.E., Kent, J.T.: Fitting smooth paths to spherical data. J. R. Stat. Soc.: Ser. C (Appl. Stat.) 36(1), 34–46 (1987)

    MathSciNet  MATH  Google Scholar 

  21. Kendall, W.S.: Probability, convexity, and harmonic maps with small image I: uniqueness and fine existence. Proceed. Lond. Math. Soc. 3(2), 371–406 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kendall, W.S.: Barycenters and hurricane trajectories. arXIV:1406.7173 (2014)

  23. Kim, T.K., Cipolla, R.: Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Trans. Pattern Anal. Mach. Intell. 31(8), 1415–1428 (2009)

    Article  Google Scholar 

  24. Kim, T.K., Wong, K.Y.K., Cipolla, R.: Tensor canonical correlation analysis for action classification. In: IEEE Conference on CVPR, pp. 1–8 (2007)

  25. Kume, A., Dryden, I.L., Le, H.: Shape-space smoothing splines for planar landmark data. Biometrika 94, 513–528 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  26. Kurtek, S., Srivastava, A., Klassen, E., Ding, Z.: Statistical modeling of curves using shapes and related features. J. Am. Stat. Assoc. 107(499), 1152–1165 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lahiri, S., Robinson, D., Klassen, E.: Precise matching of PL curves in \(R^N\) in square-root velocity framework. Geom. Imaging Comput. 2(3), 133–186 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  28. Lui, Y.M.: Human gesture recognition on product manifolds. J. Mach. Learn. Res. 13(1), 3297–3321 (2012)

    MathSciNet  MATH  Google Scholar 

  29. Lui, Y.M., Beveridge, J., Kirby, M.: Action classification on product manifolds. In: IEEE Conference on CVPR, pp. 833–839 (2010)

  30. Morris, R.J., Kent, J., Mardia, K.V., Fidrich, M., Aykroyd, R.G., Linney, A.: Analysing growth in faces. In: International conference on Imaging Science, Systems and Technology (1999)

  31. Pennec, X., Fillard, P., Ayache, N.: A Riemannian framework for tensor computing. Int. J. Comput. Vision 66(1), 41–66 (2006)

    Article  MATH  Google Scholar 

  32. Ramsay, J.O., Silverman, B.W.: Functional Data Analysis. Springer Series in Statistics. Springer, New York (2005)

    Google Scholar 

  33. Samir, C., Absil, P.A., Srivastava, A., Klassen, E.: A gradient-descent method for curve fitting on Riemannian manifolds. Found. Comput. Math. 12(1), 49–73 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  34. Schwartzman, A., Mascarenhas, W., Taylor, J.: Inference for eigenvalues and eigenvectors of Gaussian symmetric matrices. Ann. Stat. 36(6), 2886–2919 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  35. Srivastava, A., Klassen, E.: Functional and Shape Data Analysis. Springer, New York (2016)

    Book  MATH  Google Scholar 

  36. Srivastava, A., Klassen, E., Joshi, S., Jermyn, I.: Shape analysis of elastic curves in Euclidean spaces. IEEE Trans. Pattern Anal. Mach. Intell. 33(7), 1415–1428 (2011)

    Article  Google Scholar 

  37. Su, J., Dryden, I.L., Klassen, E., Le, H., Srivastava, A.: Fitting optimal curves to time-indexed, noisy observations on nonlinear manifolds. J. Image Vis. Comput. 30(6–7), 428–442 (2012)

    Article  Google Scholar 

  38. Su, J., Kurtek, S., Klassen, E., Srivastava, A.: Statistical analysis of trajectories on Riemannian manifolds: bird migration, hurricane tracking and video surveillance. Ann. Appl. Stat. 8(1), 530–552 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  39. Su, J., Srivastava, A., de Souza, F., Sarkar, S.: Rate-invariant analysis of trajectories on Riemannian manifolds with application in visual speech recognition. In: 2014 IEEE Conference on CVPR, pp. 620–627 (2014)

  40. Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: 9th European Conference on Computer Vision, pp. 589–600 (2006)

  41. Zhang, Z., Su, J., Klassen, E., Le, H., Srivastava, A.: Video-based action recognition using rate-invariant analysis of covariance trajectories. arXiv:1503.06699 (2015)

  42. Zhao, G., Barnard, M., Pietikäinen, M.: Lipreading with local spatiotemporal descriptors. IEEE Trans. Multimed. 11(7), 1254–1265 (2009)

    Article  Google Scholar 

  43. Zhao, G., Pietikäinen, M., Hadid, A.: Local spatiotemporal descriptors for visual recognition of spoken phrases. In: Proceedings of the International Workshop on Human-centered Multimedia, HCM ’07, pp. 57–66 (2007)

  44. Dryden, I.L., Koloydenko, A., Zhou, D.: Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging. Ann. Appl. Stat. 3(3), 1102–1123 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY, USA

    Zhengwu Zhang

  2. Department of Mathematics and Statistics, Texas Tech University, Lubbock, TX, USA

    Jingyong Su

  3. Department of Mathematics, Florida State University, Tallahassee, FL, USA

    Eric Klassen

  4. School of Mathematical Sciences, University of Nottingham, Nottingham, UK

    Huiling Le

  5. Department of Statistics, Florida State University, Tallahassee, FL, USA

    Anuj Srivastava

Authors
  1. Zhengwu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingyong Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric Klassen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huiling Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anuj Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengwu Zhang.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 170 KB)

Supplementary material 2 (pdf 182 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Su, J., Klassen, E. et al. Rate-Invariant Analysis of Covariance Trajectories. J Math Imaging Vis 60, 1306–1323 (2018). https://doi.org/10.1007/s10851-018-0814-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-018-0814-0

Keywords

  • SPDM Riemannian structure
  • SPDM parallel transport
  • Invariant metrics
  • Covariance trajectories
  • Vector bundles
  • Rate-invariant classification