Skip to main content
Log in

A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Human behaviour recognition has been, and still remains, a challenging problem that involves different areas of computational intelligence. The automated understanding of people activities from video sequences is an open research topic in which the computer vision and pattern recognition areas have made big efforts. In this paper, the problem is studied from a prediction point of view. We propose a novel method able to early detect behaviour using a small portion of the input, in addition to the capabilities of it to predict behaviour from new inputs. Specifically, we propose a predictive method based on a simple representation of trajectories of a person in the scene which allows a high level understanding of the global human behaviour. The representation of the trajectory is used as a descriptor of the activity of the individual. The descriptors are used as a cue of a classification stage for pattern recognition purposes. Classifiers are trained using the trajectory representation of the complete sequence. However, partial sequences are processed to evaluate the early prediction capabilities having a specific observation time of the scene. The experiments have been carried out using the three different dataset of the CAVIAR database taken into account the behaviour of an individual. Additionally, different classic classifiers have been used for experimentation in order to evaluate the robustness of the proposal. Results confirm the high accuracy of the proposal on the early recognition of people behaviours.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Angkititrakul P, Miyajima C, Takeda K (2013) Stochastic mixture modeling of driving behavior during car following. J Inf Commun Converg Eng 11(2):95–102

    Google Scholar 

  2. Antonakaki P, Kosmopoulos D, Perantonis SJ (2009) Detecting abnormal human behaviour using multiple cameras. Signal Process 89(9):1723–1738

    Article  MATH  Google Scholar 

  3. Azorin-Lopez J, Saval-Calvo M, Fuster-Guillo A, Garcia-Rodriguez J (2013) Human behaviour recognition based on trajectory analysis using neural networks. In: Neural Networks (IJCNN), The 2013 International Joint Conference on 2013. IEEE, pp 1–7

  4. Beaton P, Chen Q, Meghdir H (1996) Predictive validity in stated choice studies: a before and after comparison with revealed preference. In: 1996 IEEE international conference on systems, man and cybernetics, vol 1. Information Intelligence and Systems (Cat. No. 96CH35929). IEEE, pp 205–209

  5. Cao Y, Barrett D, Barbu A, Narayanaswamy S, Yu H, Michaux A, Lin Y, Dickinson S, Siskind JM, Wang S (2013) Recognize human activities from partially observed videos. In: 2013 IEEE conference on computer vision and pattern recognition. IEEE, pp 2658–2665

  6. Fisher R, Santos-Victor J, Crowley J (2005) CAVIAR: context aware vision using image-based active recognition project. http://homepages.inf.ed.ac.uk/rbf/CAVIAR/

  7. Fisher R, Santos-Victor J, Crowley J (2005) CAVIAR hidden semi-Markov model behaviour recognition. http://homepages.inf.ed.ac.uk/rbf/CAVIAR/hsmm.htm

  8. Hoai M, De la Torre F (2012) Max-margin early event detectors. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2863–2870

  9. Kitani KM, Ziebart BD, Bagnell JA, Herbert M (2012) Activity forecasting. In: 12th European conference on computer vision, Florence, Italy, October 7–13, 2012, Proceedings, Part IV, pp 201–214

  10. Koppula HS, Saxena A (2013) Anticipating human activities using object affordances for reactive robotic response. In: Robotics: science and systems (RSS). Berlin, Germany

  11. Lavee G, Rivlin E, Rudzsky M (2009) Understanding video events: a survey of methods for automatic interpretation of semantic occurrences in video. Syst Man Cybern Part C Appl Rev IEEE Trans 39(5):489–504

    Article  Google Scholar 

  12. Lavee G, Rudzsky M, Rivlin E, Borzin A (2010) Video event modeling and recognition in generalized stochastic petri nets. Circuits Syst Video Technol IEEE Trans 20(1):102–118

    Article  Google Scholar 

  13. Miyajima C, Angkititrakul P, Takeda K (2013) Behavior signal processing for vehicle applications. APSIPA Trans Signal Inf Process 2:e2

    Article  Google Scholar 

  14. Moeslund TB, Hilton A, Krüger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Underst 104(2–3):90–126

    Article  Google Scholar 

  15. Mori T, Takada A, Noguchi H, Harada T, Sato T (2005) Behavior prediction based on daily-life record database in distributed sensing space. In: 2005 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 1703–1709

  16. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote : synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    MATH  Google Scholar 

  17. Ryoo MS (2011) Human activity prediction: early recognition of ongoing activities from streaming videos. In: 2011 international conference on computer vision. IEEE, pp 1036–1043

  18. Saval-Calvo M, Azorín-López J, Fuster-Guilló A (2012) Comparative analysis of temporal segmentation methods of video sequences. In: Garcia-Rodriguez J, Cazorla Quevedo MA (eds) Robotic vision. IGI Global, Hershey

    Google Scholar 

  19. Schindler K, van Gool L (2008) Action snippets: how many frames does human action recognition require? In: 2008 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  20. Takano W, Imagawa H, Nakamura Y (2011) Prediction of human behaviors in the future through symbolic inference. In: 2011 IEEE international conference on robotics and automation. IEEE, pp 1970–1975

  21. Tran C, Doshi A, Trivedi MM (2012) Modeling and prediction of driver behavior by foot gesture analysis. Comput Vis Image Underst 116(3):435–445

    Article  Google Scholar 

  22. Turaga P, Chellappa R, Subrahmanian V, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video Technol 18(11):1473–1488

    Article  Google Scholar 

  23. Tweed D, Fisher R, Bins J, List T (2005) Efficient hidden semi-markov model inference for structured video sequences. In: Visual surveillance and performance evaluation of tracking and surveillance, 2005. 2nd Joint IEEE international workshop on, pp 247–254

  24. Uddin MZ, Byun KM, Cho MH, Lee SY, Khang G, Kim TS (2011) A spanning tree-based human activity prediction system using life logs from depth silhouette-based human activity recognition. Springer, Berlin

  25. Ziebart BD, Ratliff N, Gallagher G, Mertz C, Peterson K, Bagnell JA, Hebert M, Dey AK, Srinivasa S (2009) Planning-based prediction for pedestrians. In: 2009 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3931–3936

Download references

Acknowledgments

This work was supported in part by the University of Alicante, Valencian Government and Spanish government under grants GRE11-01, GV/2013/005 and DPI2013-40534-R.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jorge Azorin-Lopez.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azorin-Lopez, J., Saval-Calvo, M., Fuster-Guillo, A. et al. A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences. Neural Process Lett 43, 363–387 (2016). https://doi.org/10.1007/s11063-015-9412-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-015-9412-y

Keywords

Navigation