International Journal of Computer Vision

, Volume 116, Issue 2, pp 161–173 | Cite as

Geometry-Based Symbolic Approximation for Fast Sequence Matching on Manifolds

  • Rushil AnirudhEmail author
  • Pavan Turaga


In this paper, we consider the problem of fast and efficient indexing techniques for sequences evolving in non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search, and recognition in very high dimensional spaces. The problem is made more challenging when representations such as landmarks, contours, and human skeletons etc. are naturally studied in a non-Euclidean setting where even simple operations are much more computationally intensive than their Euclidean counterparts. We propose a geometry and data adaptive symbolic framework that is shown to enable the deployment of fast and accurate algorithms for activity recognition, dynamic texture recognition, motif discovery. Toward this end, we present generalizations of key concepts of piece-wise aggregation and symbolic approximation for the case of non-Euclidean manifolds. We show that one can replace expensive geodesic computations with much faster symbolic computations with little loss of accuracy in activity recognition and discovery applications. The framework is general enough to work across both Euclidean and non-Euclidean spaces, depending on appropriate feature representations without compromising on the ultra-low bandwidth, high speed and high accuracy. The proposed methods are ideally suited for real-time systems and low complexity scenarios.


Manifold sequence indexing Competitive learning Activity recognition Differential geometry Data mining 



Rushil Anirudh and Pavan Turaga were supported by the NSF CCF CIF grant #1320267.


  1. Absil, P.-A., Mahony, R., & Sepulchre, R. (2004). Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Applicandae Mathematicae, 80(2), 199–220.CrossRefMathSciNetzbMATHGoogle Scholar
  2. Ali, S., Basharat, A., & Shah, M. (2007). Chaotic invariants for human action recognition. In ICCV (pp. 1–8).Google Scholar
  3. Allauzen, C., & Raffinot, M. (2000). Simple optimal string matching algorithm. In Combinatorial Pattern Matching (Vol. 1848, pp. 364–374). Lecture Notes in Computer Science Berlin Heidelberg: Springer.Google Scholar
  4. Boothby, W. M. (2003). An introduction to differentiable manifolds and Riemannian geometry (2nd ed.). New York: Academic Press.Google Scholar
  5. Çetingül, H. E., & Vidal, R. (2009). Intrinsic mean shift for clustering on stiefel and grassmann manifolds. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1896–1902). Miami.Google Scholar
  6. Chakrabarti, K., Keogh, E. J., Mehrotra, S., & Pazzani, M. J. (2002). Locally adaptive dimensionality reduction for indexing large time series databases. ACM Transactions on Database System, 27(2), 188–228.CrossRefGoogle Scholar
  7. Chan, A., & Vasconcelos, N. (2005). Classification and retrieval of traffic video using auto-regressive stochastic processes. In Proceedings of IEEE Intelligent Vehicles Symposium, 2005 (pp. 771–776).Google Scholar
  8. Chaudhry, R., & Ivanov, Y. (2010). Fast approximate nearest neighbor methods for non-Euclidean manifolds with applications to human activity analysis in videos. In European Conference on Computer Vision. Crete.Google Scholar
  9. Chaudhry, R., Ravichandran, A., Hager, G., & Vidal, R. (2009). Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In CVPR, 2009 (pp. 1932–1939).Google Scholar
  10. Chum, O., Perdoch, M., & Matas, J. (2009). Geometric min-hashing: Finding a (thick) needle in a haystack. In CVPR (pp. 17–24).Google Scholar
  11. Desieno, D. (1988). Adding a conscience to competitive learning. IEEE International Conference on Neural Networks, 1, 117–124.CrossRefGoogle Scholar
  12. Devroye, L., Szpankowski, W., & Rais, B. (1992). A note on the height of suffix trees. SIAM Journal on Computing, 21(1), 48–53.CrossRefMathSciNetzbMATHGoogle Scholar
  13. Fletcher, P. T., Lu, C., Pizer, S. M., & Joshi, S. C. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging, 23(8), 995–1005.CrossRefGoogle Scholar
  14. Gaur, U., Zhu, Y., Song, B., & Chowdhury, A. K. R. (2011). A ‘string of feature graphs’ model for recognition of complex activities in natural videos. In ICCV (pp. 2595–2602).Google Scholar
  15. Goodall, C. R., & Mardia, K. V. (1999). Projective shape analysis. Journal of Computational and Graphical Statistics, 8(2), 143–198.MathSciNetGoogle Scholar
  16. Gorelick, L., Blank, M., Shechtman, E., Irani, M., & Basri, R. (2007). Actions as space-time shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2247–2253.CrossRefGoogle Scholar
  17. Grove, K., & Karcher, H. (1973). How to conjugate C\(^1\)-close group actions. Mathematische Zeitschrift, 132, 11–20.CrossRefMathSciNetzbMATHGoogle Scholar
  18. Harandi, M. T., Salzmann, M., & Hartley, R. (2014). From manifold to manifold: Geometry-aware dimensionality reduction for SPD matrices. In ECCV (pp. 17–32).Google Scholar
  19. Jordan, M. I. (1998). Learning in Graphical Models. Cambridge: MIT Press.CrossRefzbMATHGoogle Scholar
  20. Joshi, S. H., Klassen, E., Srivastava, A., & Jermyn, I. (2007). A novel representation for Riemannian analysis of elastic curves in R\(^{\text{ n }}\). In CVPR.Google Scholar
  21. Kendall, D. (1984). Shape manifolds, procrustean metrics and complex projective spaces. Bulletin of London Mathematical society, 16, 81–121.CrossRefMathSciNetzbMATHGoogle Scholar
  22. Kohonen, T. (1995). Self-organizing maps. Berlin: Springer.CrossRefGoogle Scholar
  23. Lafferty, J. D., & Lebanon, G. (2005). Diffusion kernels on statistical manifolds. Journal of Machine Learning Research, 6, 129–163.MathSciNetzbMATHGoogle Scholar
  24. Lin, J., Keogh, E. J., Lonardi, S., & chi Chiu, B. Y. (2003). A symbolic representation of time series, with implications for streaming algorithms. In DMKD (pp. 2–11).Google Scholar
  25. Lin, J., & Li, Y. (2010). Finding approximate frequent patterns in streaming medical data. In CBMS (pp. 13–18).Google Scholar
  26. Lin, T., & Zha, H. (2008). Riemannian manifold learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 796–809.CrossRefGoogle Scholar
  27. Lui, Y. M., Beveridge, J. R., & Kirby, M. (2010). Action classification on product manifolds. In CVPR (pp. 833–839).Google Scholar
  28. Mueen, A., Keogh, E. J., Zhu, Q., Cash, S., & Westover, M. B. (2009). Exact discovery of time series motifs. In SDM (pp. 473–484).Google Scholar
  29. Patel, P., Keogh, E., Lin, J., & Lonardi, S. (2002). Mining motifs in massive time series databases. In Proceedings of the 2002 IEEE International Conference on Data Mining, ICDM 2003 (pp. 370–377).Google Scholar
  30. Pennec, X. (2006). Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measurements. Journal of Mathematical Imaging and Vision, 25(1), 127–154.CrossRefMathSciNetGoogle Scholar
  31. Pennec, X., Fillard, P., & Ayache, N. (2006). A Riemannian framework for tensor computing. International Journal of Computer Vision, 66(1), 41–66.CrossRefMathSciNetzbMATHGoogle Scholar
  32. Rahman, I. U., Drori, I., Stodden, V. C., Donoho, D. L., & Schrder, P. (2005). Multiscale representations for manifold-valued data. Multiscale Modeling and Simulation, 4(4), 1201–1232.CrossRefMathSciNetzbMATHGoogle Scholar
  33. Revaud, J., Douze, M., Schmid, C., & Jegou, H. (2013). Event retrieval in large video collections with circulant temporal encoding. In CVPR (pp. 2459–2466).Google Scholar
  34. Ripley, B. D. (1996). Pattern recognition and neural networks. Cambridge: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  35. Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.CrossRefGoogle Scholar
  36. Sankaranarayanan, A. C., Turaga, P. K., Baraniuk, R. G., & Chellappa, R. (2010). Compressive acquisition of dynamic scenes. In ECCV (vol. 1, pp. 129–142).Google Scholar
  37. Soatto, S., Doretto, G., & Wu, Y. N. (2001). Dynamic textures. ICCV (vol. 2, pp. 439–446).Google Scholar
  38. Spivak, M. (1999). A comprehensive introduction to differential geometry (3rd ed., Vol. 1). Houston: Publish or Perish Inc.zbMATHGoogle Scholar
  39. Srivastava, A., Jermyn, I., & Joshi, S. (2007). Riemannian analysis of probability density functions with applications in vision. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1–8).Google Scholar
  40. Srivastava, A., Klassen, E., Joshi, S. H., & Jermyn, I. H. (2011). Shape analysis of elastic curves in Euclidean spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 1415–1428.CrossRefGoogle Scholar
  41. Su, J., Kurtek, S., Klassen, E., & Srivastava, A. (2014). Statistical analysis of trajectories on Riemannian manifolds: Bird migration, hurricane tracking, and video surveillance. Annals of Applied Statistics, 8(1), 530–552.CrossRefMathSciNetzbMATHGoogle Scholar
  42. Tenenbaum, J. B., Silva, Vd, & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2319–2323.CrossRefGoogle Scholar
  43. Turaga, P., Veeraraghavan, A., Srivastava, A., & Chellappa, R. (2010). Statistical analysis on manifolds and its applications to video analysis. In D. Schonfeld, C. Shan, D. Tao, & L. Wang (Eds.), Video search and mining (Vol. 287, pp. 115–144)., Studies in computational intelligence Berlin: Springer.CrossRefGoogle Scholar
  44. Turaga, P. K., & Chellappa, R. (2009). Locally time-invariant models of human activities using trajectories on the Grassmannian. In CVPR (pp. 2435–2441).Google Scholar
  45. Turaga, P. K., Veeraraghavan, A., Srivastava, A., & Chellappa, R. (2011). Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(11), 2273–2286.CrossRefGoogle Scholar
  46. Tuzel, O., Porikli, F. M., & Meer, P. (2006). Region covariance: A fast descriptor for detection and classification. In European Conference on Computer Vision (vol. II, pp. 589–600).Google Scholar
  47. Vahdatpour, A., Amini, N., & Sarrafzadeh, M. (2009). Toward unsupervised activity discovery using multi-dimensional motif detection in time series. In IJCAI (pp. 1261–1266).Google Scholar
  48. Veeraraghavan, A., & Chowdhury, A. K. R. (2006). The function space of an activity. In CVPR (vol. 1, pp. 959–968).Google Scholar
  49. Veeraraghavan, A., Chowdhury, A. K. R., & Chellappa, R. (2005). Matching shape sequences in video with applications in human movement analysis. IEEE Transactions on Pattern Analysis and Machine Intellience, 27(12), 1896–1909.CrossRefGoogle Scholar
  50. Vemulapalli, R., Arrate, F., & Chellappa, R. (2014). Human action recognition by representing 3d skeletons as points in a lie group. In CVPR, 2014 (pp. 588–595).Google Scholar
  51. Vishwanathan, S. V. N., Borgwardt, K. M., Kondor, I. R., & Schraudolph, N. N. (2008). Graph kernels. CoRR, abs/0807.0093.Google Scholar
  52. Xia, L., Chen, C., & Aggarwal, J. (2012). View invariant human action recognition using histograms of 3d joints. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012. IEEE (pp. 20–27).Google Scholar
  53. Yao, A. (1979). The complexity of pattern matching for a random string. SIAM Journal on Computing, 8(3), 368–387.CrossRefMathSciNetzbMATHGoogle Scholar
  54. Yi, S., Krim, H., & Norris, L. K. (2012). Human activity as a manifold-valued random process. IEEE Transactions on Image Processing, 21(8), 3416–3428.CrossRefMathSciNetGoogle Scholar
  55. Zador, P. (1982). Asymptotic quantization error of continuous signals and the quantization dimension. IEEE Transactions on Information Theory, 28(2), 139–149.CrossRefMathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.School of Electrical, Computer and Energy EngineeringArizona State UniversityTempeUSA
  2. 2.School of Arts, Media and EngineeringArizona State UniversityTempeUSA

Personalised recommendations