Human Action Recognition Using Histograms of Oriented Optical Flows from Depth

  • Baris Can Ustundag
  • Mustafa Unel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8887)


In this paper we develop a new method for recognizing human actions from depth data. 2D optical flows from depth images are computed for the entire action instance. From the resulting optical flow vectors, patches are defined around each joint location to learn local motion variations. These patches are grouped in terms of their joints and used to extract a new feature called ‘Histograms of Oriented Optical Flows from Depth (HOOFD)’. In order to encode temporal variations, these features are generated in a pyramidal fashion. At each level of the pyramid, action instance is partitioned equally into two parts and each part is employed separately to compute histograms. Oriented optical flow histograms are utilized due to their invariance to scale and direction of motion. We performed several experiments on publicly available databases and compared our approach with some of the state-of-the-art methods. Results show the success of the proposed method.


Optical Flow Action Recognition Depth Image Action Instance Linear Support Vector Machine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aggarwal, J., Ryoo, M.: Human activity analysis: A review. ACM Comput. 43, 16:1–16:43 (2011)Google Scholar
  2. 2.
    Weinland, D., Ronfard, R., Boyer, E.: A survey of vision-based methods for action representation, segmentation and recognition. Comput. Vis. Image Underst. 115, 224–241 (2011)CrossRefGoogle Scholar
  3. 3.
    Dollar, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: ICCCN 2005, pp. 65–72 (2005)Google Scholar
  4. 4.
    Laptev, I.: On space-time interest points. Int. J. Comput. Vision 64, 107–123 (2005)CrossRefGoogle Scholar
  5. 5.
    Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: ICCV, pp. 1395–1402 (2005)Google Scholar
  6. 6.
    Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE Trans. Pattern Anal. Mach. Intell. 23, 257–267 (2001)CrossRefGoogle Scholar
  7. 7.
    Eichner, M., Marin-Jimenez, M., Zisserman, A., Ferrari, V.: 2d articulated human pose estimation and retrieval in (almost) unconstrained still images. International Journal of Computer Vision 99, 190–214 (2012)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Li, W., Zhang, Z., Liu, Z.: Action recognition based on a bag of 3d points. In: Computer Vision and Pattern Recognition Workshops, CVPRW (2010)Google Scholar
  9. 9.
    Oreifej, O., Liu, Z.: Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences. In: CVPR, pp. 716–723 (2013)Google Scholar
  10. 10.
    Xia, L., Aggarwal, J.K.: Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. In: CVPR, pp. 2834–2841 (2013)Google Scholar
  11. 11.
    Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A.: Efficient human pose estimation from single depth images. In: Pattern Analysis and Machine Intellingence (2013)Google Scholar
  12. 12.
    Yang, X., Tian, Y.: Effective 3d action recognition using eigenjoints. J. Vis. Comun. Image Represent., 2–11 (2014)Google Scholar
  13. 13.
    Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., Bajcsy, R.: Sequence of the most informative joints (smij): A new representation for human skeletal action recognition. In: Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 8–13 (2012)Google Scholar
  14. 14.
    Wang, J., Liu, Z., Wu, Y., Yuan, J.: Learning actionlet ensemble for 3d human action recognition. In: Pattern Analysis and Machine Intelligence, pp. 914–927 (2014)Google Scholar
  15. 15.
    Chaudhry, R., Ravichandran, A., Hager, G.D., Vidal, R.: Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: CVPR, pp. 1932–1939 (2009)Google Scholar
  16. 16.
    Müller, M., Röder, T.: Motion templates for automatic classification and retrieval of motion capture data. In: Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 137–146 (2006)Google Scholar
  17. 17.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intellingence 17, 185–203 (1981)CrossRefGoogle Scholar
  18. 18.
    Lv, F., Nevatia, R.: Recognition and segmentation of 3-D human action using HMM and multi-class adaBoost. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 359–372. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Baris Can Ustundag
    • 1
  • Mustafa Unel
    • 1
  1. 1.Faculty of Engineering and Natural SciencesSabanci UniversityIstanbulTurkey

Personalised recommendations