Skip to main content
SpringerLink
Log in

Human action recognition in videos based on the Transferable Belief Model

Application to athletics jumps

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

This paper focuses on human behavior recognition where the main problem is to bridge the semantic gap between the analogue observations of the real world and the symbolic world of human interpretation. For that, a fusion architecture based on the Transferable Belief Model framework is proposed and applied to action recognition of an athlete in video sequences of athletics meeting with moving camera. Relevant features are extracted from videos, based on both the camera motion analysis and the tracking of particular points on the athlete’s silhouette. Some models of interpretation are used to link the numerical features to the symbols to be recognized, which are running, jumping and falling actions. A Temporal Belief Filter is then used to improve the robustness of action recognition. The proposed approach demonstrates good performance when tested on real videos of athletics sports videos (high jumps, pole vaults, triple jumps and long jumps) acquired by a moving camera and different view angles. The proposed system is also compared to Bayesian Networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Many other papers are available on Smets’ homepage http://iridia.ulb.ac.be/~psmets. The web page also proposes links towards other researchers in the TBM community as well as softwares.

  2. The software can bedownloaded onhttp://www.irisa.fr/Vista/Motion2D.

  3. In the first version, 18 points aretracked; but in this paper, image quality is not sufficient for such a level ofdetail (which is not useful here).

  4. The notion of distinctness is close, but notequivalent to the independence notion in probability theory. See [43] for more details.

  5. Here, the frame is an image butnot a FoD.

  6. De Morgan’s algebra can also beapplied.

  7. A focal element corresponds to a propositionfor which the belief is not null.

  8. A focal set in a BBA is a set for which the associated beliefmass is not null.

  9. When the BBAs are defined on a FoD witha cardinality greater than 2, the decision must not be taken using beliefmasses, but pignistic probabilities [52].

References

  1. Hu W, Tan T, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviors. IEEE Trans Systems Man Cybern C 34(3):334–352

    Article  Google Scholar 

  2. Messer K, Christmas WJ, Jaser E, Kittler J, Levienaise-Obadia B, Koubaroulis D (2005) A unified approach to the generation of semantic cues for sports video annotation. Signal Processing 85:357–383

    Article  Google Scholar 

  3. Jaimes A, Sebe N (2005) Multimodal human computer interaction: a survey. In: IEEE International Workshop on Human Computer Interaction in conjunction with ICCV, vol 3766. Beijing, China, pp 1–15

  4. Li B, Errico JH, Pan H, Sezan I (2004) Bridging the semantic gap in sports video retrieval and summarization. J Vis Commun Image Represent 15:393–424

    Google Scholar 

  5. Sadlier DA, O’Connor NE (2005) Event detection in field sports video using audio-visual features and a support vector machine. IEEE Trans Circuit Syst Video Technol 15(10):1225–1233

    Google Scholar 

  6. Lew M, Sebe N, Eakins J (2002) Challenges in image and video retrieval. Lect Notes Comput Sci ICIVR 2383:1–6

    Article  Google Scholar 

  7. Freitas ND, Brochu E, Barnard K, Duygulu P, Forsyth D (2002) Bayesian models for massive multimedia databases: a new frontier. In: Valencia International Meeting on Bayesian Statistics/2002 ISBA International Meeting

  8. Shah M (2003) Understanding human behavior from motion imagery. Mach Vis Appl 14:210–214

    Article  Google Scholar 

  9. Hongeng S, Nevatia R, Bremond F (2004) Video-based event recognition and probabilistic recognition methods. Comput Vis Image Underst 96:129–162

    Article  Google Scholar 

  10. Klir GJ, Wierman MJ (1999) Uncertainty-based information. Elements of generalized information theory, 2nd edn. Studies in fuzzyness and soft computing. Physica-Verlag, New York

  11. Dubois D, Grabisch M, Prade H, Smets Ph (2001) Using the transferable belief model and a qualitative possibility theory approach on an illustrative example: the assessment of the value of a candidate. Int J Intell Syst 16:1245–1272

    Article  MATH  Google Scholar 

  12. Ristic B, Smets P (2005) Target identification using belief functions and implication rules. IEEE Trans Aerosp Electron Syst 41(3):1097–1102

    Article  Google Scholar 

  13. Petkovic M, Jonker W (2004) Integrated use of different content derivation techniques within a multimedia database management system. J Vis Commun Image Represent 15:303–329

    Article  Google Scholar 

  14. Smets P, Kennes R (1994) The transferable belief model. Artif Intell 66(2):191–234

    Article  MATH  MathSciNet  Google Scholar 

  15. Shafer G (1976) A mathematical theory of evidence. Princeton University Press, Princeton

    MATH  Google Scholar 

  16. Smets P (1994) Advances in the Dempster–Shafer theory of evidence—What is Dempster–Shafer’s model? In: Yager RR, Fedrizzi M, Kacprzyk J, edition. Wiley, New York, pp 5–34

  17. Ramasso E, Rombaut M, Pellerin D (2006) A temporal belief filter improving human action recognition in videos. In: IEEE international conference on acoustics, speech and signal processing, vol 2, pp 141–144

  18. Schubert J (2004) Clustering belief functions based on attracting and conflicting metalevel evidence using Potts spin mean field theory. Inform Fusion 5(4):309–318

    Article  MathSciNet  Google Scholar 

  19. Wang L, Hu W, Tan T (2003) Recent developments in human motion analysis. Pattern Recognit 36(3):585–601

    Article  Google Scholar 

  20. Yacoob Y, Black M (1999) Parametrized modeling and recognition of activities. Comput Vis Image Underst 73(2):232–247

    Article  Google Scholar 

  21. Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77:257–285

    Article  Google Scholar 

  22. Murphy KP (2002) Dynamic Bayesian Networks: representation, inference and learning. PhD thesis, UC Berkeley (CSD)

  23. Xiang T, Gong S (2003) Discovering Bayesian causality among visual events in a complex outdoor scene. In: IEEE on advanced video and signal based surveillance, pp 177–182

  24. Luo Y, Wu TD, Hwang JN (2003) Object-based analysis and interpretation of human motion in sports video sequences by dynamic Bayesian networks. Comput Vis Image Underst 92:196–216

    Article  Google Scholar 

  25. Vefghi L, Linkens DA (1999) Dynamic monitoring and control of patient anaesthetic and dose levels: time-delay, moving-average neural networks, and principal components analysis. Comput Methods Programs Biomed 59:91–106

    Article  Google Scholar 

  26. Zhong D, Chang S-F (2004) Real-time view recognition and event detection for sports video. J Vis Commun Image Represent 15:330–347

    Article  Google Scholar 

  27. Medioni G, Cohen I, Brémond F, Hongeng S, Nevatia R (2001) Event detection and analysis from video streams. PAMI 23(8):873–889

    Google Scholar 

  28. Ayers D, Shah M (2001) Monitoring human behavior from video taken in an office environment. Image Vis Comput 13:833–846

    Article  Google Scholar 

  29. Ding Z, Bunke H, Schneider M, Kandel A (2005) Fuzzy timed Petri net: definitions, properties, and applications. Math Comput Model 41:345–360

    Article  MATH  MathSciNet  Google Scholar 

  30. Lee SJ, Seong PH (2004) Development of automated operating procedure system using fuzzy colored Petri nets for nuclear power plants. Ann Nucl Energy 31:849–869

    Article  Google Scholar 

  31. Fay A (2000) A fuzzy knowledge-based system for railway traffic control. Eng Appl Artif Intell 13:719–729

    Article  Google Scholar 

  32. Bourbakis N, Gattiker JR, Bebis G (2003) Interpreting a dynamic and uncertain world: task-based control. Int J Artif Intell Tools 12(1):5–85

    Article  Google Scholar 

  33. Nigro JM, Loriette-Rougegrez S, Rombaut M (2002) Driving situation recognition with uncertainty management and rule-based systems. Eng Appl Artif Intell 15:217–228

    Article  Google Scholar 

  34. Rombaut M, Jarkass I, Denoeux T (1999) State recognition in discret dynamical systems using Petri nets and evidence theory. In: European conference on symbolic and quantitative approaches to reasoning with uncertainty

  35. Girondel V, Caplier A, Bonnaud L, Rombaut M (2005) Belief theory-based classifiers comparison for static human body postures recognition in video. Int J Signal Process 2(1):29–33

    Google Scholar 

  36. Hammal Z, Caplier A, Rombaut M (2005) Belief theory applied to facial expressions classification. In: International conference on advances in pattern recognition, Bath, UK

  37. Moeslund TB, Granum E (2001) A survey of computer vision-based human motion capture. Comput Vis Image Underst 81:231–268

    Article  MATH  Google Scholar 

  38. Wang J, Singh S (2003) Video analysis of human dynamics—a survey. Real-Time Imaging 9(5):321–346

    Article  Google Scholar 

  39. Odobez JM, Bouthemy P (1995) Robust multiresolution estimation of parametric motion models. J Vis Commun Image Represent 6(4):348–365

    Article  Google Scholar 

  40. Piriou G, Bouthemy P, Peyrard N, Yao JF (2002) Probabilistic models of image motion for recognition of dynamic content in video. In: International workshop on computer vision and image analysis, vol 11. Las Palmas de Gran Canaria, Spain

  41. Fablet R, Bouthemy P, Perez P (2002) Non parametric motion characterization using causal probabilistic models for video indexing and retrieval. IEEE Trans Image Process 11(4):393–407

    Article  Google Scholar 

  42. Panagiotakis C, Tziritas G (2004) Recognition and tracking of the members of a moving human body. In: Articulated motion and deformable objects, pp 86–98

  43. Yaghlane B, Smets P, Mellouli K (2002) Independence concept for belief functions. In: Technologies for constructing intelligent systems: tools, Heidelberg, Germany. Physica-Verlag GmbH, New York

  44. Ayoun A, Smets Ph (2001) Data association in multi-target detection using the transferable belief model. Int J Intell Syst 16(10):1167–1182

  45. Zribi M, Benjelloun M (2003) Parametric estimation of Dempster–Shafer belief functions. In: International conference on information fusion, pp 485–491

  46. Elouedi Z, Mellouli K, Smets Ph (2004) Assessing sensor reliability for multisensor data fusion within the transferable belief model. IEEE Trans Syst Man Cybern 34(1):782–787

    Article  Google Scholar 

  47. Smets P (1993) Beliefs functions: the disjunctive rule of combination and the Generalized Bayesian Theorem. Int J Approx Reason 9:1–35

    Article  MATH  MathSciNet  Google Scholar 

  48. Rombaut M, Zhu YM (2002) Study of Dempster–Shafer theory for image segmentation applications. Image Vis Comput 20(1):15–23

    Article  Google Scholar 

  49. Denoeux T, Yaghlane AB (2002) Approximating the combination of belief functions using the fast moebius transform in a coarsened frame. Int J Approx Reason 37:77–101

    Article  Google Scholar 

  50. Philippe Smets (2002) The application of the matrix calculus to belief functions. Int J Approx Reason 31(1–2):1–30

    Article  MATH  MathSciNet  Google Scholar 

  51. Haenni R, Lehmann N (2003) Implementing belief function computations. Int J Intell Syst 18(1):31–49

    Article  MATH  Google Scholar 

  52. Smets Ph (2005) Decision making in the TBM: the necessity of the pignistic transformation. Int J Approx Reason 38:133–147

    Article  MATH  MathSciNet  Google Scholar 

  53. Ramasso E, Pellerin D, Rombaut M (2006) Belief scheduling for the recognition of human action sequence. In: International conference on information fusion, Florence, Italia, pp 1–8

  54. Panagiotakis C, Ramasso E, Tziritas G, Rombaut M, Pellerin D (2006) Shape-motion based athlete tracking for multilevel action recognition. In: Perales FJ, Fisher RB (eds) Proceedings of the fourth international conference on articulated motion and deformable objects. Springer, Berlin, pp 385–394

  55. Witten IH, Frank E (2005) Data-mining: practical machine learning tools and techniques, 2nd edn. Morgan Kaufman, San Francisco, Publishers, pp 560. ISBN 0-12-088407-0

Download references

Acknowledgments

This research is partially supported by SIMILAR European excellence network. The authors thank the Vista research team at Irisa/Inria Rennes (France) for the use of the Motion2D software.

Author information

Authors and Affiliations

  1. DIS Department, GIPSA-Lab, 46 avenue Félix Viallet, 38031, Grenoble, France

    E. Ramasso, D. Pellerin & M. Rombaut

  2. Department of Computer Science, University of Crete, P.O. Box 2208, Heraklion, Greece

    C. Panagiotakis

Authors
  1. E. Ramasso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Panagiotakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Pellerin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Rombaut
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. Ramasso.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ramasso, E., Panagiotakis, C., Pellerin, D. et al. Human action recognition in videos based on the Transferable Belief Model. Pattern Anal Applic 11, 1–19 (2008). https://doi.org/10.1007/s10044-007-0073-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-007-0073-y

Keywords

Search

Navigation