Mode-Driven Volume Analysis Based on Correlation of Time Series

  • Chengcheng JiaEmail author
  • Wei Pang
  • Yun Fu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8925)


Tensor analysis is widely used for face recognition and action recognition. In this paper, a mode-driven discriminant analysis (MDA) in tensor subspace is proposed for visual recognition. For training, we treat each sample as an N-order tensor, of which the first N-1 modes capture the spatial information of images while the N-th mode captures the sequential patterns of images. We employ Fisher criteria on the first N-1 modes to extract discriminative features of the visual information. After that, considering the correlation of adjacent frames in the sequence, i.e., the current frame and its former and latter ones, we update the sequence by calculating the correlation of triple adjacent frames, then perform discriminant analysis on the N-th mode. The alternating projection procedure of MDA converges and is convex with different initial values of the transformation matrices. Such hybrid tensor subspace learning scheme may sufficiently preserve both discrete and continuous distributions information of action videos in lower dimensional spaces to boost discriminant power. Experiments on the MSR action 3D database, KTH database and ETH database showed that our algorithm MDA outperformed other tensor-based methods in terms of accuracy and is competitive considering the time efficiency. Besides, it is robust to deal with the damaged and self-occluded action silhouettes and RGB object images in various viewing angles.


Action Recognition Canonical Correlation Analysis Adjacent Frame Human Action Recognition Gait Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ballani, J., Grasedyck, L.: A projection method to solve linear systems in tensor format. Numerical Linear Algebra with Applications 20(1), 27–43 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE TPAMI 19(7), 711–720 (1997)CrossRefGoogle Scholar
  3. 3.
    Birnbaum, A., Johnstone, I.M., Nadler, B., Paul, D., et al.: Minimax bounds for sparse pca with noisy high-dimensional data. The Annals of Statistics 41(3), 1055–1084 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Biswas, S., Aggarwal, G., Flynn, P.J., Bowyer, K.W.: Pose-robust recognition of low-resolution face images. TPAMI 35(12), 3037–3049 (2013)CrossRefGoogle Scholar
  5. 5.
    Fukunaga, K.: Introduction to statistical pattern recognition. Pattern Recognition 22(7), 833–834 (1990)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Gong, D., Medioni, G.: Dynamic manifold warping for view invariant action recognition. In: ICCV, pp. 571–578. IEEE (2011)Google Scholar
  7. 7.
    Gong, W., Sapienza, M., Cuzzolin, F.: Fisher tensor decomposition for unconstrained gait recognition. Training 2, 3 (2013)Google Scholar
  8. 8.
    Grasedyck, L., Kressner, D., Tobler, C.: A literature survey of low-rank tensor approximation techniques. GAMM-Mitteilungen 36(1), 53–78 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Guo, K., Ishwar, P., Konrad, J.: Action recognition from video using feature covariance matrices. IEEE TIP 22(6), 2479–2494 (2013)MathSciNetGoogle Scholar
  10. 10.
    Ho, H.T., Gopalan, R.: Model-driven domain adaptation on product manifolds for unconstrained face recognition. IJCV 109(1–2), 110–125 (2014)CrossRefzbMATHGoogle Scholar
  11. 11.
    Hu, H.: Enhanced gabor feature based classification using a regularized locally tensor discriminant model for multiview gait recognition. IEEE Transactions on Circuits and Systems for Video Technology 23(7), 1274–1286 (2013)CrossRefGoogle Scholar
  12. 12.
    Huang, C.-H., Yeh, Y.-R., Wang, Y.-C.F.: Recognizing actions across cameras by exploring the correlated subspace. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part I. LNCS, vol. 7583, pp. 342–351. Springer, Heidelberg (2012) Google Scholar
  13. 13.
    Jia, C., Wang, S., Peng, X., Pang, W., Zhang, C., Zhou, C., Yu, Z.: Incremental multi-linear discriminant analysis using canonical correlations for action recognition. Neurocomputing 83, 56–63 (2012)CrossRefGoogle Scholar
  14. 14.
    Jia, C., Zhong, G., Fu, Y.: Low-rank tensor learning with discriminant analysis for action classification and image recovery. In: AAAI (2014)Google Scholar
  15. 15.
    Kim, T., Cipolla, R.: Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE T. Pattern Anal. 1415–1428 (2008)Google Scholar
  16. 16.
    Kolda, T., Bader, B.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    de Laat, K.F., van Norden, A.G., Gons, R.A., van Oudheusden, L.J., van Uden, I.W., Norris, D.G., Zwiers, M.P., de Leeuw, F.E.: Diffusion tensor imaging and gait in elderly persons with cerebral small vessel disease. Stroke 42(2), 373–379 (2011)CrossRefGoogle Scholar
  18. 18.
    Leibe, B., Schiele, B.: Analyzing appearance and contour based methods for object categorization. In: CVPR, vol. 2, pp. II-409 (2003)Google Scholar
  19. 19.
    Lui, Y.M., Beveridge, J.R.: Tangent bundle for human action recognition. In: FG, pp. 97–102. IEEE (2011)Google Scholar
  20. 20.
    Lykou, A., Whittaker, J.: Sparse cca using a lasso with positivity constraints. Computational Statistics & Data Analysis 54(12), 3144–3157 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Miyamoto, K., Adachi, Y., Osada, T., Watanabe, T., Kimura, H.M., Setsuie, R., Miyashita, Y.: Dissociable memory traces within the macaque medial temporal lobe predict subsequent recognition performance. The Journal of Neuroscience 34(5), 1988–1997 (2014)CrossRefGoogle Scholar
  22. 22.
    Goud Tandarpally, M., Nagendar, G., Ganesh Bandiatmakuri, S., Jawahar, C.V.: Action recognition using canonical correlation kernels. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part III. LNCS, vol. 7726, pp. 479–492. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  23. 23.
    Perez, E.A., Mota, V.F., Maciel, L.M., Sad, D., Vieira, M.B.: Combining gradient histograms using orientation tensors for human action recognition. In: ICPR, pp. 3460–3463. IEEE (2012)Google Scholar
  24. 24.
    Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: ICPR, vol. 3, pp. 32–36 (2004)Google Scholar
  25. 25.
    Tao, D., Li, X., Wu, X., Maybank, S.: General tensor discriminant analysis and gabor features for gait recognition. IEEE T. Pattern Anal. 29, 1700–1715 (2007)CrossRefGoogle Scholar
  26. 26.
    Tian, C., Fan, G., Gao, X., Tian, Q.: Multiview face recognition: From tensorface to v-tensorface and k-tensorface. IEEE T. Syst. Man Cy. B 42(2), 320–333 (2012)CrossRefGoogle Scholar
  27. 27.
    Wu, X., Wang, H., Liu, C., Jia, Y.: Cross-view action recognition over heterogeneous feature spaces. In: ICCV, pp. 609–616 (2013)Google Scholar
  28. 28.
    Xue, G., Mei, L., Chen, C., Lu, Z.L., Poldrack, R., Dong, Q.: Spaced learning enhances subsequent recognition memory by reducing neural repetition suppression. Journal of Cognitive Neuroscience 23(7), 1624–1633 (2011)CrossRefGoogle Scholar
  29. 29.
    Yan, S., Xu, D., Yang, Q., Zhang, L., Tang, X., Zhang, H.: Discriminant analysis with tensor representation. In: CVPR, vol. 1, pp. 526–532 (2005)Google Scholar
  30. 30.
    Yang, F., Bourdev, L., Shechtman, E., Wang, J., Metaxas, D.: Facial expression editing in video using a temporally-smooth factorization. In: CVPR, pp. 861–868. IEEE (2012)Google Scholar
  31. 31.
    Youn, J., Cho, J.W., Lee, W.Y., Kim, G.M., Kim, S.T., Kim, H.T.: Diffusion tensor imaging of freezing of gait in patients with white matter changes. Movement Disorders 27(6), 760–764 (2012)CrossRefGoogle Scholar
  32. 32.
    Yu, Z.Z., Jia, C.C., Pang, W., Zhang, C.Y., Zhong, L.H.: Tensor discriminant analysis with multiscale features for action modeling and categorization. IEEE Signal Processing Letters 19(2), 95–98 (2012)CrossRefGoogle Scholar
  33. 33.
    Zafeiriou, S.: Discriminant nonnegative tensor factorization algorithms. IEEE TNN 20(2), 217–235 (2009)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Electrical and Computer EngineeringNortheastern UniversityBostonUSA
  2. 2.Computer and Information ScienceNortheastern UniversityBostonUSA
  3. 3.School of Natural and Computing SciencesUniversity of AberdeenAberdeenUK

Personalised recommendations