Sparse Code Filtering for Action Pattern Mining

  • Wei WangEmail author
  • Yan Yan
  • Liqiang Nie
  • Luming Zhang
  • Stefan Winkler
  • Nicu Sebe
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10112)


Action recognition has received increasing attention during the last decade. Various approaches have been proposed to encode the videos that contain actions, among which self-similarity matrices (SSMs) have shown very good performance by encoding the dynamics of the video. However, SSMs become sensitive when there is a very large view change. In this paper, we tackle the multi-view action recognition problem by proposing a sparse code filtering (SCF) framework which can mine the action patterns. First, a class-wise sparse coding method is proposed to make the sparse codes of the between-class data lie close by. Then we integrate the classifiers and the class-wise sparse coding process into a collaborative filtering (CF) framework to mine the discriminative sparse codes and classifiers jointly. The experimental results on several public multi-view action recognition datasets demonstrate that the presented SCF framework outperforms other state-of-the-art methods.


Action Recognition Sparse Code Collaborative Filter Dictionary Learning Video Descriptor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Cai, Z., Wang, L., Peng, X., Qiao, Y.: Multi-view super vector for action recognition. In: CVPR (2014)Google Scholar
  2. 2.
    Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3D skeletons as points in a lie group. In: CVPR (2014)Google Scholar
  3. 3.
    Lv, F., Nevatia, R.: Single view human action recognition using key pose matching and viterbi path searching. In: CVPR (2007)Google Scholar
  4. 4.
    Peursum, P., Venkatesh, S., West, G.: Tracking-as-recognition for articulated full-body human motion analysis. In: CVPR (2007)Google Scholar
  5. 5.
    Farhadi, A., Tabrizi, M.K.: Learning to recognize activities from the wrong view point. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 154–166. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88682-2_13 CrossRefGoogle Scholar
  6. 6.
    Junejo, I.N., Dexter, E., Laptev, I., Perez, P.: View-independent action recognition from temporal self-similarities. TPAMI 33(1), 172–185 (2011)CrossRefGoogle Scholar
  7. 7.
    Weinland, D., Boyer, E., Ronfard, R.: Action recognition from arbitrary views using 3D exemplars. In: ICCV (2007)Google Scholar
  8. 8.
    Natarajan, P., Nevatia, R.: View and scale invariant action recognition using multiview shape-flow models. In: CVPR (2008)Google Scholar
  9. 9.
    Yan, Y., Ricci, E., Subramanian, R., Liu, G., Sebe, N.: Multitask linear discriminant analysis for view invariant action recognition. TIP 23(12), 5599–5611 (2014)MathSciNetGoogle Scholar
  10. 10.
    Mahasseni, B., Todorovic, S.: Latent multitask learning for view-invariant action recognition. In: ICCV (2013)Google Scholar
  11. 11.
    Matikainen, P., Sukthankar, R., Hebert, M.: Model recommendation for action recognition. In: CVPR (2012)Google Scholar
  12. 12.
    Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: CVPR (2015)Google Scholar
  13. 13.
    Wang, W., Cui, Z., Yan, Y., Feng, J., Yan, S., Shu, X., Sebe, N.: Recurrent face aging. In: CVPR (2016)Google Scholar
  14. 14.
    Wang, W., Tulyakov, S., Sebe, N.: Recurrent convolutional face alignment. In: ACCV (2016)Google Scholar
  15. 15.
    Junejo, I.N., Dexter, E., Laptev, I., Pérez, P.: Cross-view action recognition from temporal self-similarities. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5303, pp. 293–306. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88688-4_22 CrossRefGoogle Scholar
  16. 16.
    Sun, C., Junejo, I., Foroosh, H.: Action recognition using rank-1 approximation of joint self-similarity volume. In: ICCV (2011)Google Scholar
  17. 17.
    Wang, W., Yan, Y., Winkler, S., Sebe, N.: Category specific dictionary learning for attribute specific feature selection. TIP 25(3), 1465–1478 (2016)MathSciNetGoogle Scholar
  18. 18.
    Wang, W., Yan, Y., Sebe, N.: Attribute guided dictionary learning. In: ICMR (2015)Google Scholar
  19. 19.
    Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning: transfer learning from unlabeled data. In: ICML (2007)Google Scholar
  20. 20.
    Luo, J., Wang, W., Qi, H.: Group sparsity and geometry constrained dictionary learning for action recognition from depth maps. In: ICCV (2013)Google Scholar
  21. 21.
    Qiu, Q., Jiang, Z., Chellappa, R.: Sparse dictionary-based representation and recognition of action attributes. In: ICCV (2011)Google Scholar
  22. 22.
    Guha, T., Ward, R.K.: Learning sparse representations for human action recognition. TPAMI 34(8), 1576–1588 (2012)CrossRefGoogle Scholar
  23. 23.
    Zheng, J., Jiang, Z.: Learning view-invariant sparse representations for cross-view action recognition. In: ICCV (2013)Google Scholar
  24. 24.
    Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using collaborative filtering to weave an information tapestry. Commun. ACM 35(12), 61–70 (1992)CrossRefGoogle Scholar
  25. 25.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Goldberg, K., Roeder, T., Gupta, D., Perkins, C.: Eigentaste: a constant time collaborative filtering algorithm. Inf. Retrieval 4(2), 133–151 (2001)CrossRefzbMATHGoogle Scholar
  27. 27.
    Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes. CVIU 104(2), 249–257 (2006)Google Scholar
  28. 28.
    Weinland, D., Özuysal, M., Fua, P.: Making action recognition robust to occlusions and viewpoint changes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 635–648. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15558-1_46 CrossRefGoogle Scholar
  29. 29.
    Huang, C.-H., Yeh, Y.-R., Wang, Y.-C.F.: Recognizing actions across cameras by exploring the correlated subspace. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7583, pp. 342–351. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33863-2_34 CrossRefGoogle Scholar
  30. 30.
    Li, R., Zickler, T.: Discriminative virtual views for cross-view action recognition. In: CVPR (2012)Google Scholar
  31. 31.
    Liu, J., Shah, M.: Learning human actions via information maximization. In: CVPR (2008)Google Scholar
  32. 32.
    Reddy, K.K., Liu, J., Shah, M.: Incremental action recognition using feature-tree. In: ICCV (2009)Google Scholar
  33. 33.
    Baumann, F., Ehlers, A., Rosenhahn, B., Liao, J.: Recognizing human actions using novel space-time volume binary patterns. Neurocomputing 173, 54–63 (2016)CrossRefGoogle Scholar
  34. 34.
    Ashraf, N., Sun, C., Foroosh, H.: View invariant action recognition using projective depth. CVIU 123, 41–52 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Wei Wang
    • 1
    Email author
  • Yan Yan
    • 1
  • Liqiang Nie
    • 2
  • Luming Zhang
    • 4
  • Stefan Winkler
    • 3
  • Nicu Sebe
    • 1
  1. 1.University of TrentoTrentoItaly
  2. 2.National University of SingaporeSingaporeSingapore
  3. 3.Advanced Digital Sciences CenterSingaporeSingapore
  4. 4.Hefei University of TechnologyHefeiChina

Personalised recommendations