Abstract
In this paper, we consider the problem of fast and efficient indexing techniques for sequences evolving in non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search, and recognition in very high dimensional spaces. The problem is made more challenging when representations such as landmarks, contours, and human skeletons etc. are naturally studied in a non-Euclidean setting where even simple operations are much more computationally intensive than their Euclidean counterparts. We propose a geometry and data adaptive symbolic framework that is shown to enable the deployment of fast and accurate algorithms for activity recognition, dynamic texture recognition, motif discovery. Toward this end, we present generalizations of key concepts of piece-wise aggregation and symbolic approximation for the case of non-Euclidean manifolds. We show that one can replace expensive geodesic computations with much faster symbolic computations with little loss of accuracy in activity recognition and discovery applications. The framework is general enough to work across both Euclidean and non-Euclidean spaces, depending on appropriate feature representations without compromising on the ultra-low bandwidth, high speed and high accuracy. The proposed methods are ideally suited for real-time systems and low complexity scenarios.
Similar content being viewed by others
References
Absil, P.-A., Mahony, R., & Sepulchre, R. (2004). Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Applicandae Mathematicae, 80(2), 199–220.
Ali, S., Basharat, A., & Shah, M. (2007). Chaotic invariants for human action recognition. In ICCV (pp. 1–8).
Allauzen, C., & Raffinot, M. (2000). Simple optimal string matching algorithm. In Combinatorial Pattern Matching (Vol. 1848, pp. 364–374). Lecture Notes in Computer Science Berlin Heidelberg: Springer.
Boothby, W. M. (2003). An introduction to differentiable manifolds and Riemannian geometry (2nd ed.). New York: Academic Press.
Çetingül, H. E., & Vidal, R. (2009). Intrinsic mean shift for clustering on stiefel and grassmann manifolds. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1896–1902). Miami.
Chakrabarti, K., Keogh, E. J., Mehrotra, S., & Pazzani, M. J. (2002). Locally adaptive dimensionality reduction for indexing large time series databases. ACM Transactions on Database System, 27(2), 188–228.
Chan, A., & Vasconcelos, N. (2005). Classification and retrieval of traffic video using auto-regressive stochastic processes. In Proceedings of IEEE Intelligent Vehicles Symposium, 2005 (pp. 771–776).
Chaudhry, R., & Ivanov, Y. (2010). Fast approximate nearest neighbor methods for non-Euclidean manifolds with applications to human activity analysis in videos. In European Conference on Computer Vision. Crete.
Chaudhry, R., Ravichandran, A., Hager, G., & Vidal, R. (2009). Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In CVPR, 2009 (pp. 1932–1939).
Chum, O., Perdoch, M., & Matas, J. (2009). Geometric min-hashing: Finding a (thick) needle in a haystack. In CVPR (pp. 17–24).
Desieno, D. (1988). Adding a conscience to competitive learning. IEEE International Conference on Neural Networks, 1, 117–124.
Devroye, L., Szpankowski, W., & Rais, B. (1992). A note on the height of suffix trees. SIAM Journal on Computing, 21(1), 48–53.
Fletcher, P. T., Lu, C., Pizer, S. M., & Joshi, S. C. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging, 23(8), 995–1005.
Gaur, U., Zhu, Y., Song, B., & Chowdhury, A. K. R. (2011). A ‘string of feature graphs’ model for recognition of complex activities in natural videos. In ICCV (pp. 2595–2602).
Goodall, C. R., & Mardia, K. V. (1999). Projective shape analysis. Journal of Computational and Graphical Statistics, 8(2), 143–198.
Gorelick, L., Blank, M., Shechtman, E., Irani, M., & Basri, R. (2007). Actions as space-time shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2247–2253.
Grove, K., & Karcher, H. (1973). How to conjugate C\(^1\)-close group actions. Mathematische Zeitschrift, 132, 11–20.
Harandi, M. T., Salzmann, M., & Hartley, R. (2014). From manifold to manifold: Geometry-aware dimensionality reduction for SPD matrices. In ECCV (pp. 17–32).
Jordan, M. I. (1998). Learning in Graphical Models. Cambridge: MIT Press.
Joshi, S. H., Klassen, E., Srivastava, A., & Jermyn, I. (2007). A novel representation for Riemannian analysis of elastic curves in R\(^{\text{ n }}\). In CVPR.
Kendall, D. (1984). Shape manifolds, procrustean metrics and complex projective spaces. Bulletin of London Mathematical society, 16, 81–121.
Kohonen, T. (1995). Self-organizing maps. Berlin: Springer.
Lafferty, J. D., & Lebanon, G. (2005). Diffusion kernels on statistical manifolds. Journal of Machine Learning Research, 6, 129–163.
Lin, J., Keogh, E. J., Lonardi, S., & chi Chiu, B. Y. (2003). A symbolic representation of time series, with implications for streaming algorithms. In DMKD (pp. 2–11).
Lin, J., & Li, Y. (2010). Finding approximate frequent patterns in streaming medical data. In CBMS (pp. 13–18).
Lin, T., & Zha, H. (2008). Riemannian manifold learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 796–809.
Lui, Y. M., Beveridge, J. R., & Kirby, M. (2010). Action classification on product manifolds. In CVPR (pp. 833–839).
Mueen, A., Keogh, E. J., Zhu, Q., Cash, S., & Westover, M. B. (2009). Exact discovery of time series motifs. In SDM (pp. 473–484).
Patel, P., Keogh, E., Lin, J., & Lonardi, S. (2002). Mining motifs in massive time series databases. In Proceedings of the 2002 IEEE International Conference on Data Mining, ICDM 2003 (pp. 370–377).
Pennec, X. (2006). Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measurements. Journal of Mathematical Imaging and Vision, 25(1), 127–154.
Pennec, X., Fillard, P., & Ayache, N. (2006). A Riemannian framework for tensor computing. International Journal of Computer Vision, 66(1), 41–66.
Rahman, I. U., Drori, I., Stodden, V. C., Donoho, D. L., & Schrder, P. (2005). Multiscale representations for manifold-valued data. Multiscale Modeling and Simulation, 4(4), 1201–1232.
Revaud, J., Douze, M., Schmid, C., & Jegou, H. (2013). Event retrieval in large video collections with circulant temporal encoding. In CVPR (pp. 2459–2466).
Ripley, B. D. (1996). Pattern recognition and neural networks. Cambridge: Cambridge University Press.
Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.
Sankaranarayanan, A. C., Turaga, P. K., Baraniuk, R. G., & Chellappa, R. (2010). Compressive acquisition of dynamic scenes. In ECCV (vol. 1, pp. 129–142).
Soatto, S., Doretto, G., & Wu, Y. N. (2001). Dynamic textures. ICCV (vol. 2, pp. 439–446).
Spivak, M. (1999). A comprehensive introduction to differential geometry (3rd ed., Vol. 1). Houston: Publish or Perish Inc.
Srivastava, A., Jermyn, I., & Joshi, S. (2007). Riemannian analysis of probability density functions with applications in vision. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1–8).
Srivastava, A., Klassen, E., Joshi, S. H., & Jermyn, I. H. (2011). Shape analysis of elastic curves in Euclidean spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 1415–1428.
Su, J., Kurtek, S., Klassen, E., & Srivastava, A. (2014). Statistical analysis of trajectories on Riemannian manifolds: Bird migration, hurricane tracking, and video surveillance. Annals of Applied Statistics, 8(1), 530–552.
Tenenbaum, J. B., Silva, Vd, & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2319–2323.
Turaga, P., Veeraraghavan, A., Srivastava, A., & Chellappa, R. (2010). Statistical analysis on manifolds and its applications to video analysis. In D. Schonfeld, C. Shan, D. Tao, & L. Wang (Eds.), Video search and mining (Vol. 287, pp. 115–144)., Studies in computational intelligence Berlin: Springer.
Turaga, P. K., & Chellappa, R. (2009). Locally time-invariant models of human activities using trajectories on the Grassmannian. In CVPR (pp. 2435–2441).
Turaga, P. K., Veeraraghavan, A., Srivastava, A., & Chellappa, R. (2011). Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(11), 2273–2286.
Tuzel, O., Porikli, F. M., & Meer, P. (2006). Region covariance: A fast descriptor for detection and classification. In European Conference on Computer Vision (vol. II, pp. 589–600).
Vahdatpour, A., Amini, N., & Sarrafzadeh, M. (2009). Toward unsupervised activity discovery using multi-dimensional motif detection in time series. In IJCAI (pp. 1261–1266).
Veeraraghavan, A., & Chowdhury, A. K. R. (2006). The function space of an activity. In CVPR (vol. 1, pp. 959–968).
Veeraraghavan, A., Chowdhury, A. K. R., & Chellappa, R. (2005). Matching shape sequences in video with applications in human movement analysis. IEEE Transactions on Pattern Analysis and Machine Intellience, 27(12), 1896–1909.
Vemulapalli, R., Arrate, F., & Chellappa, R. (2014). Human action recognition by representing 3d skeletons as points in a lie group. In CVPR, 2014 (pp. 588–595).
Vishwanathan, S. V. N., Borgwardt, K. M., Kondor, I. R., & Schraudolph, N. N. (2008). Graph kernels. CoRR, abs/0807.0093.
Xia, L., Chen, C., & Aggarwal, J. (2012). View invariant human action recognition using histograms of 3d joints. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012. IEEE (pp. 20–27).
Yao, A. (1979). The complexity of pattern matching for a random string. SIAM Journal on Computing, 8(3), 368–387.
Yi, S., Krim, H., & Norris, L. K. (2012). Human activity as a manifold-valued random process. IEEE Transactions on Image Processing, 21(8), 3416–3428.
Zador, P. (1982). Asymptotic quantization error of continuous signals and the quantization dimension. IEEE Transactions on Information Theory, 28(2), 139–149.
Acknowledgments
Rushil Anirudh and Pavan Turaga were supported by the NSF CCF CIF grant #1320267.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Larry Davis.
Rights and permissions
About this article
Cite this article
Anirudh, R., Turaga, P. Geometry-Based Symbolic Approximation for Fast Sequence Matching on Manifolds. Int J Comput Vis 116, 161–173 (2016). https://doi.org/10.1007/s11263-015-0835-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-015-0835-8