Abstract
Human behaviour recognition has been, and still remains, a challenging problem that involves different areas of computational intelligence. The automated understanding of people activities from video sequences is an open research topic in which the computer vision and pattern recognition areas have made big efforts. In this paper, the problem is studied from a prediction point of view. We propose a novel method able to early detect behaviour using a small portion of the input, in addition to the capabilities of it to predict behaviour from new inputs. Specifically, we propose a predictive method based on a simple representation of trajectories of a person in the scene which allows a high level understanding of the global human behaviour. The representation of the trajectory is used as a descriptor of the activity of the individual. The descriptors are used as a cue of a classification stage for pattern recognition purposes. Classifiers are trained using the trajectory representation of the complete sequence. However, partial sequences are processed to evaluate the early prediction capabilities having a specific observation time of the scene. The experiments have been carried out using the three different dataset of the CAVIAR database taken into account the behaviour of an individual. Additionally, different classic classifiers have been used for experimentation in order to evaluate the robustness of the proposal. Results confirm the high accuracy of the proposal on the early recognition of people behaviours.
Similar content being viewed by others
References
Angkititrakul P, Miyajima C, Takeda K (2013) Stochastic mixture modeling of driving behavior during car following. J Inf Commun Converg Eng 11(2):95–102
Antonakaki P, Kosmopoulos D, Perantonis SJ (2009) Detecting abnormal human behaviour using multiple cameras. Signal Process 89(9):1723–1738
Azorin-Lopez J, Saval-Calvo M, Fuster-Guillo A, Garcia-Rodriguez J (2013) Human behaviour recognition based on trajectory analysis using neural networks. In: Neural Networks (IJCNN), The 2013 International Joint Conference on 2013. IEEE, pp 1–7
Beaton P, Chen Q, Meghdir H (1996) Predictive validity in stated choice studies: a before and after comparison with revealed preference. In: 1996 IEEE international conference on systems, man and cybernetics, vol 1. Information Intelligence and Systems (Cat. No. 96CH35929). IEEE, pp 205–209
Cao Y, Barrett D, Barbu A, Narayanaswamy S, Yu H, Michaux A, Lin Y, Dickinson S, Siskind JM, Wang S (2013) Recognize human activities from partially observed videos. In: 2013 IEEE conference on computer vision and pattern recognition. IEEE, pp 2658–2665
Fisher R, Santos-Victor J, Crowley J (2005) CAVIAR: context aware vision using image-based active recognition project. http://homepages.inf.ed.ac.uk/rbf/CAVIAR/
Fisher R, Santos-Victor J, Crowley J (2005) CAVIAR hidden semi-Markov model behaviour recognition. http://homepages.inf.ed.ac.uk/rbf/CAVIAR/hsmm.htm
Hoai M, De la Torre F (2012) Max-margin early event detectors. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2863–2870
Kitani KM, Ziebart BD, Bagnell JA, Herbert M (2012) Activity forecasting. In: 12th European conference on computer vision, Florence, Italy, October 7–13, 2012, Proceedings, Part IV, pp 201–214
Koppula HS, Saxena A (2013) Anticipating human activities using object affordances for reactive robotic response. In: Robotics: science and systems (RSS). Berlin, Germany
Lavee G, Rivlin E, Rudzsky M (2009) Understanding video events: a survey of methods for automatic interpretation of semantic occurrences in video. Syst Man Cybern Part C Appl Rev IEEE Trans 39(5):489–504
Lavee G, Rudzsky M, Rivlin E, Borzin A (2010) Video event modeling and recognition in generalized stochastic petri nets. Circuits Syst Video Technol IEEE Trans 20(1):102–118
Miyajima C, Angkititrakul P, Takeda K (2013) Behavior signal processing for vehicle applications. APSIPA Trans Signal Inf Process 2:e2
Moeslund TB, Hilton A, Krüger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Underst 104(2–3):90–126
Mori T, Takada A, Noguchi H, Harada T, Sato T (2005) Behavior prediction based on daily-life record database in distributed sensing space. In: 2005 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 1703–1709
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote : synthetic minority over-sampling technique. J Artif Intell Res 16:321–357
Ryoo MS (2011) Human activity prediction: early recognition of ongoing activities from streaming videos. In: 2011 international conference on computer vision. IEEE, pp 1036–1043
Saval-Calvo M, Azorín-López J, Fuster-Guilló A (2012) Comparative analysis of temporal segmentation methods of video sequences. In: Garcia-Rodriguez J, Cazorla Quevedo MA (eds) Robotic vision. IGI Global, Hershey
Schindler K, van Gool L (2008) Action snippets: how many frames does human action recognition require? In: 2008 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8
Takano W, Imagawa H, Nakamura Y (2011) Prediction of human behaviors in the future through symbolic inference. In: 2011 IEEE international conference on robotics and automation. IEEE, pp 1970–1975
Tran C, Doshi A, Trivedi MM (2012) Modeling and prediction of driver behavior by foot gesture analysis. Comput Vis Image Underst 116(3):435–445
Turaga P, Chellappa R, Subrahmanian V, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video Technol 18(11):1473–1488
Tweed D, Fisher R, Bins J, List T (2005) Efficient hidden semi-markov model inference for structured video sequences. In: Visual surveillance and performance evaluation of tracking and surveillance, 2005. 2nd Joint IEEE international workshop on, pp 247–254
Uddin MZ, Byun KM, Cho MH, Lee SY, Khang G, Kim TS (2011) A spanning tree-based human activity prediction system using life logs from depth silhouette-based human activity recognition. Springer, Berlin
Ziebart BD, Ratliff N, Gallagher G, Mertz C, Peterson K, Bagnell JA, Hebert M, Dey AK, Srinivasa S (2009) Planning-based prediction for pedestrians. In: 2009 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3931–3936
Acknowledgments
This work was supported in part by the University of Alicante, Valencian Government and Spanish government under grants GRE11-01, GV/2013/005 and DPI2013-40534-R.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Azorin-Lopez, J., Saval-Calvo, M., Fuster-Guillo, A. et al. A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences. Neural Process Lett 43, 363–387 (2016). https://doi.org/10.1007/s11063-015-9412-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-015-9412-y