Technical Gestures Recognition by Set-Valued Hidden Markov Models with Prior Knowledge

  • Yann Soullard
  • Alessandro Antonucci
  • Sébastien Destercke
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 456)


Hidden Markov models are popular tools for gesture recognition. Once the generative processes of gestures have been identified, an observation sequence is usually classified as the gesture having the highest likelihood, thus ignoring possible prior information. In this paper, we consider two potential improvements of such methods: the inclusion of prior information, and the possibility of considering convex sets of probabilities (in the likelihoods and the prior) to infer imprecise, but more reliable, predictions when information is insufficient. We apply the proposed approach to technical gestures, typically characterized by severe class imbalance. By modelling such imbalances as a prior information, we achieve more accurate results, while the imprecise quantification is shown to produce more reliable estimates.


Hide Markov Model Gesture Recognition Probability Mass Function Multivariate Time Series Imprecise Probability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work is founded by the European Union and the French region Picardie. Europe acts in Picardie with the European Regional Development Fund (ERDF). This work is supported by the ANR UML-net project, grant ANR-14-CE24-0026 of the French Agence Nationale de la Recherche (ANR).


  1. 1.
    Antonucci A, de Rosa R, Giusti A, Cuzzolin F (2015) Robust classification of multivariate time series by imprecise hidden Markov models. Int J Approx Reason 56(B):249–263Google Scholar
  2. 2.
    Bevilacqua F, Zamborlin B, Sypniewski A, Schnell N, Guédy F, Rasamimanana N (2010) Continuous Realtime Gesture Following and Recognition. In: Gesture in embodied communication and human-computer interaction: 8th international gesture workshop, GW 2009, Revised Selected Papers. Springer, pp 73–84Google Scholar
  3. 3.
    Bouchard G, Triggs B (2004) The tradeoff between generative and discriminative classifiers. In: International symposium on computational statistics, pp 721–728Google Scholar
  4. 4.
    Chaudhary A, Raheja JL, Das K, Raheja S (2011) Intelligent approaches to interact with machines using hand gesture recognition in natural way: a survey. Int J Comput Sci Eng Surv 2(1):122–133CrossRefGoogle Scholar
  5. 5.
    Gorelick L, Blank M, Shechtman E, Irani M, Basri R (2007) Actions as space-time shapes. IEEE Trans Pattern Anal Mach Intell 29(12):2247–2253CrossRefGoogle Scholar
  6. 6.
    Liu K, Chen C, Jafari R, Kehtarnavaz N (2014) Multi-HMM classification for hand gesture recognition using two differing modality sensors. In: Circuits and systems conference (DCAS). IEEE, pp 1–4Google Scholar
  7. 7.
    Mauá DD, Antonucci A, de Campos CP (2015) Hidden Markov models with set-valued parameters. Neurocomputing 180:94–107CrossRefGoogle Scholar
  8. 8.
    Neverova N, Wolf C, Taylor GW, Nebout F (2014) Multi-scale deep learning for gesture detection and localization. In: ECCV workshop on looking at peopleGoogle Scholar
  9. 9.
    Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286CrossRefGoogle Scholar
  10. 10.
    Soullard Y, Saveski M, Artières T (2014) Joint semi-supervised learning of hidden conditional random fields and hidden Markov models. Pattern Recogn Lett 37:161–171CrossRefGoogle Scholar
  11. 11.
    Walley P (1996) Inferences from multinomial data: learning about a bag of marbles. J R Stat Soc B 58(1):3–57MathSciNetzbMATHGoogle Scholar
  12. 12.
    Zaffalon M, Corani G, Mauá DD (2012) Evaluating credal classifiers by utility-discounted predictive accuracy. Int J Approx Reason 53(8):1282–1301MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Yann Soullard
    • 1
    • 3
  • Alessandro Antonucci
    • 2
  • Sébastien Destercke
    • 1
    • 3
  1. 1.Heudiasyc lab, CNRS UMR 7253 Heudiasyc, CS 60 319, 60 203Université de technologie deCompiègne cedexFrance
  2. 2.Istituto Dalle Molle di Studi Sull’Intelligenza ArtificialeManno-LuganoSwitzerland
  3. 3.Sorbonne UniversityParisFrance

Personalised recommendations