Abstract
In this paper, we have focused on the view-based spatio-temporal template matching approach for human action detection and classification. We have proposed an approach for human activity modeling that describes human motions as a texture pattern. We have combined two relatively simple feature extractors for obtaining a system to get more accurate result. In this method, video sequences are converted into temporal templates called Motion History Image (MHI), which can preserve dominant motion information. The local features are described with Local Binary Pattern (LBP) and Histogram of Oriented Gradients (HOG) descriptors. LBP operator is texture operator that encodes the direction of motion from the non-monotonous areas of MHI images. HOG is used as feature descriptor and extracts the features from LBP. These descriptors are used to train with Support Vector Machine (SVM) classifier to recognize various action classes. This proposed method has been tested on the KTH Action Dataset (which is one of the most widely used benchmark datasets for human action classification), and on the Pedestrian Action Dataset. Our method has shown 86.67 % recognition rate in the 6-classes of KTH Action Dataset and 94.3 % accuracy in the 7-classes of Pedestrian Action Dataset. Based on the complexity of datasets, both the results are quite satisfactory.
Similar content being viewed by others
References
Ahad MAR (2011) “Computer vision and action recognition: a guide for image processing and computer vision community for action understanding”, 1st edn. Atlantis Ambient and Pervasive Intelligence and Springer, Berlin
Ahad MAR (2013) “Motion history images for action recognition and understanding”, 1st edn. Springer, Berlin
Ahad MAR, Tan JK, Kim H, Ishikawa S (2008) “Human activity recognition: various paradigms”. International Conference in Control, Automation and Systems, pp 1896–1901
Schuldt C, Laptev I, Caputo B (2004) “Recognizing human actions: a local SVM approach”. International Conference on Pattern Recognition, pp 32–36
Mueid RM, Ahmed C, Ahad MAR (2015) “Pedestrian activity classification using patterns of motion and histogram of oriented gradient”. J Multimodal User Interfaces 1–7. doi:10.1007/s12193-015-0178-3
Turaga P, Chellappa R, Subrahmanian VS, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video Technol 18(11):1473–1488
Moeslund TB, Granum E (2001) A survey of computer vision-based human motion capture. Comput Vis Image Understand 106:90–126
Wang L, Hu W, Tan T (2003) Recent developments in human motion analysis. Pattern Recognit 36:585–601
Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Understand 108:4–18
Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28:976–990
Moeslund TB, Hilton A, Kruger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Understand 104(2–3):90–126
Gavrila DM (1999) “The visual analysis of human movement: a survey”. Comput Vis Image Understand 73:82–98
Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Understand 73(3):428–440
Bobick A, Davis J (1996) “An Appearance-based Representation of Action”. International Conference on Pattern Recognition, pp 307–312
Davis JW (1996) “Appearance-based motion recognition of human actions”, M.I.T. Media Lab Perceptual Computing Group Tech. Report No. 387, p 51
Anderson C, Bert P, Wal GV (1985) Change detection and tracking using pyramids transformation techniques. SPIE-Intell Robot Comput Vis 579:72–78
Lipton AJ, Fujiyoshi H, Patil RS (1998) “Moving Target classification and tracking from real-time video”. IEEE Workshop on Applications of Computer Vision, pp 8–14
Ahad MAR, Tan JK, Kim H, Ishikawa S (2009) Temporal motion recognition and segmentation approach. Int J Imaging Syst Technol 19:91–99
Ahad MAR, Ogata T, Tan JK, Kim H, Ishikawa S (2008) A complex motion recognition technique employing directional motion templates. Int J Innov Comput, Inf Control 4(8):1943–1954
Gavrila DM (2000) Pedestrian detection form a moving vehicle, vol 1843. Springer, Berlin. pp 37–49
Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans on PAMI 27(10):1615–1630
Dalal N, Triggs B (2005) “Histograms of oriented gradients for human detection”. International Conference on Computer Vision and Pattern Recognition, pp 886–893
Bobick A, Davis J (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267
Meng H, Pears N, Bailey C (2007) “A human action recognition system for embedded computer vision application”. In: Workshop on Embedded Computer Vision (with CVPR), pp 1–6
Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans PAMI 28(2):316–322
Chandrashekhar V, Venkatesh KS (2006) “Action energy images for reliable human action recognition”. In: Asian Symposium on Information Display (ASID), pp 484–487
Liu J, Zhang N (2007) “Gait history image: a novel temporal template for gait recognition”. IEEE International Conference on Multimedia and Expo, pp 663–666
Ahad MAR, Tan J, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23(2):255–281
Ojala T, Pietikainen M, Harwood D (1994) Performance evaluation of texture measures with classification based on kullback discrimination of distributions. Int Conf Pattern Recognit 1:582–585
Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recognit 29(1):51–59
Pietikainen M, Zhao G, Hadid A, Ahonen T (2011) “Computer vision using local binary pattern”, 1st edn. Springer, Berlin
Kellokumpu V, Zhao G, Pietikainen M (2008) Texture based description of movements for activity analysis. Int Conf Comput Vis Theory Appl 1:206–213
Heikkila J, Pietikainen M (2006) A texture-based method for modeling the background and detecting moving objects. IEEE Trans PAMI 28(4):657–662
Heikkila J, Pietikainen M (2004) “A texture-based method for detecting moving objects”. British Machine Vision Conference, pp 187–196
Hadid A, Pietikainen M (2009) Combining appearance and motion for face and gender recognition from videos. Pattern Recognit 42(11):2818–2827
Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans PAMI 29(6):915–928
Huang C, Hsieh C, Lai K, Huang WY (2011) “Human action recognition using histogram of oriented gradient of motion history image”. In: International conference on instrumentation, measurement, computer, communication and control, pp 353–356
Bertozzi M, Broggi A, Del Rose M, Felisa M, Rakotomamonjy A, Suard F (2007) “A pedestrian detector using histograms of oriented gradients and a support vector machine classifier”. In: IEEE intelligent transportation systems conference, pp 143–144
Dalal N, Triggs B, Schmid C (2006) “Human detection using oriented histograms of flow and appearance”. In: European Conference on Computer Vision, pp 428–441
Osuna E, Freund R, Girosi F (1997) “Training support vector machines: an application to face detection”. CVPR, pp 130–136
Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans neural Netw 10(5):988–999
Weinland D, Ronfard R, Boyer E (2006) Free viewpoint action recognition using motion history volumes. Comput Vis Image Understand 104:249–257
Wolf L, Shashua A (2003) “Kernel principal angles for classification machines with applications to image sequence interpretation”. CVPR, pp 635–640
Wallraven C, Caputo B, Graf A (2003) “Recognition with local features: the kernel recipe”. ICCV, pp 257–264
Klaser A, Marszalek M, Schmid C (2008) “A spatio-temporal descriptor based on 3D gradients”. British machine vision conference
Gilbert A, Illingworth J, Bowden R (2011) Action recognition using mined hierarchical compound features. IEEE Trans Pattern Anal Mach Intell 33(5): 883–897. doi:10.1109/TPAMI.2010.144
Niebles J, Wang H, Fei-Fei L (2006) “Unsupervised learning of human action categories using spatial-temporal words”. In: British machine vision conference
Dollar P, Rabaud V, Cottrell G, Belongie S (2005) “Behavior recognition via sparse spatiotemporal features”. In: International workshop on visual surveillance and performance evaluation of tracking and surveillance, pp 65–72
Wong S, Cipolla R, “Extracting spatio-temporal interest points using global information”. In: International conference on computer vision
Bregonzio M, Li J, Gong S, Xiang T (2010) “Discriminative topics modeling for action feature selection and recognition”. In: British machine vision conference
Acknowledgments
Authors are thankful to the Center for Natural Science and Engineering Research (CNSER) and UGC, Bangladesh.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rahman Ahad, M., Islam, M.N. & Jahan, I. Action recognition based on binary patterns of action-history and histogram of oriented gradient. J Multimodal User Interfaces 10, 335–344 (2016). https://doi.org/10.1007/s12193-016-0229-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12193-016-0229-4