Human Action Recognition Using Key Points Displacement

  • Kuan-Ting Lai
  • Chaur-Heh Hsieh
  • Mao-Fu Lai
  • Ming-Syan Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6134)


Recognizing human actions is currently one of the most active research topics. Efros et al. first proposed using optical flow and normalized correlation to recognize distant actions. One weakness of the method is that optical flow is too noisy and cannot reveal the true motions; the other popular method is the space-time-interest-points proposed by Laptev et al., who extended the Harris corner detector to temporal domain. Inspired by the two methods, we proposed a new algorithm based on displacement of Lowe’s scale-invariant key points to detect motions. The vectors of matched key points are calculated as weighted orientation histograms and then classified by SVM. Experimental results demonstrate that the proposed motion descriptor is effective on recognizing both general and sport actions.


SIFT Action Recognition Optical Flow Space-time-interest-points SVM 


  1. 1.
    Laptev, I.: On space-time interest points. IJCV 64(2-3), 107–123 (2005)CrossRefGoogle Scholar
  2. 2.
    Schuldt, C., Laptev, I., Caputo, B.: Recognizing Human Actions: A Local SVM Approach. In: Proc. ICPR (2004)Google Scholar
  3. 3.
    Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: ICCV workshop: VS-PETS (2005)Google Scholar
  4. 4.
    Niebles, J.C., Wang, H.C., Li, F.F.: Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words. In: Proc. BMCV (2006)Google Scholar
  5. 5.
    Wong, S.-F., Kim, T., Cipolla, R.: Learning Motion Categories using both Semantics and Structural Information. In: CVPR (2007)Google Scholar
  6. 6.
    Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal on Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  7. 7.
    Harris, C., Stephens, M.J.: A combined corner and edge detector. In: Alvey Vision Conference, pp. 147–152 (1988)Google Scholar
  8. 8.
    Schmid, C., Mohr, R., Bauckhage., C.: Evaluation of interest point detectors. IJCV 37(2), 151–172 (2000)zbMATHCrossRefGoogle Scholar
  9. 9.
    Liu, J., Shah, M.: Learning human actions via information maximization. In: CVPR (2008)Google Scholar
  10. 10.
    Efros, A.A., Berg, A., Mori, G., Malik, J.: Recognizing Action at a Distance. In: International Conference on Computer Vision (2003)Google Scholar
  11. 11.
    Hofmann, T.: Probabilistic Latent Semantic Indexing. In: Proceedings of the Twenty-Second Annual. International SIGIR Conference on Research and Development in Information Retrieval, SIGIR-99 (1999)Google Scholar
  12. 12.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet Allocation. Journal of Machine Learning Research 3, 993–1022 (2003) doi:10.1162/jmlr.2003.3.4-5.993zbMATHCrossRefGoogle Scholar
  13. 13.
    Harris, C., Stephens, M.J.: A combined corner and edge detector. In: Alvey Vision Conference, pp. 147–152 (1988)Google Scholar
  14. 14.
    Ke, Y., Sukthankar, R., Hebert, M.: Efficient Visual Event Detection using Volumetric Features. In: International Conference on Computer Vision, pp. 166–173 (2005)Google Scholar
  15. 15.
    Scovanner, P., Ali, S., Shah, M.: A 3-dimensional SIFT descriptor and its application to action recognition. In: Proceedings of the 15th International Conference on Multimedia, pp. 357–360 (2007)Google Scholar
  16. 16.
    Sun, X.H., Chen, M.Y.: Action Recognition via Local Descriptors and Holistic Features. In: CVPR (2009)Google Scholar
  17. 17.
    Hess, R.: SIFT Feature Detector,
  18. 18.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001)Google Scholar
  19. 19.
    Wang, H., Ullah, M.M., Klaser, A., Laptev, I.: Evaluation of Local Spatio-temporal Features for Action Recognition. In: CVPR (2009)Google Scholar
  20. 20.
    Bobick, A., Davis, J.: The Representation and Recognition of Action Using Temporal Templates. PAMI 23(3), 257–267 (2001)Google Scholar
  21. 21.
    Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as Space-Time Shapes. IEEE PAMI 29(12), 2247–2253 (2007)Google Scholar
  22. 22.
    Yilmaz, A., Shah, M.: A Differential Geometric Approach to Representing the Human Actions. Comput. Vis. Image Und. 109(3), 335–351 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Kuan-Ting Lai
    • 1
    • 2
  • Chaur-Heh Hsieh
    • 3
  • Mao-Fu Lai
    • 4
  • Ming-Syan Chen
    • 1
    • 2
  1. 1.Research Center for Information Technology InnovationAcademia SinicaTaiwan
  2. 2.National Taiwan UniversityTaipeiTaiwan, R.O.C.
  3. 3.Ming-Chuan UniversityTaoyuanTaiwan, R.O.C.
  4. 4.Tungnan UniversityTaipeiTaiwan, R.O.C.

Personalised recommendations