Inertial Gesture Recognition with BLSTM-RNN

  • Grégoire LefebvreEmail author
  • Samuel Berlemont
  • Franck Mamalet
  • Christophe Garcia
Conference paper
Part of the Springer Series in Bio-/Neuroinformatics book series (SSBN, volume 4)


This chapter presents a new robust method for inertialMEM (MicroElectroMechanical systems) based 3D gesture recognition. The linear acceleration and the angular velocity, respectively provided by the accelerometer and the gyrometer, are sampled in time resulting in 6D values at each timestep, which are used as inputs for our gesture recognition system. We propose to build a system based on Bidirectional Long Short-Term Memory Recurrent Neural Networks (BLSTMRNN) for gesture classification from raw MEM data. We compare this system to a statistical method based on HMM (Hidden Markov Model), to a geometric approach using DTW (Dynamic Time Warping), and to a specific classifier FDSVM (Frame-based Descriptor and multi-class Support Vector Machine) using filtered and denoisedMEM data. Experimental results, on a dataset composed of 22 individuals producing 14 gestures, show that the proposed approach outperforms classical methods with an average classification rate of 95.57% and a standard deviation of 0.50 for 616 test gestures. Furthermore, these experiments underline that combining accelerometer and gyrometer data gives better results than using a single inertial description.


Gesture Recognition Hand Gesture Recognition Dynamic Gesture Gesture Recognition System Correct Recognition Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akl, A., Valaee, S.: Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing. In: ICASSP (2010)Google Scholar
  2. 2.
    Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., Baskurt, A.: Sequential Deep Learning for Human Action Recognition. In: Salah, A.A., Lepri, B. (eds.) HBU 2011. LNCS, vol. 7065, pp. 29–39. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., Baskurt, A.: Spatio-Temporal Convolutional Sparse Auto-Encoder for Sequence Classification. In: BMVC (2012)Google Scholar
  4. 4.
    Cho, S.J., Choi, E., Bang, W.C., Yang, J., Sohn, J., Kim, D.Y., Lee, Y.B., Kim, S.: Two-stage Recognition of Raw Acceleration Signals for 3-D Gesture-Understanding Cell Phones. In: Lorette, G. (ed.) Tenth International Workshop on Frontiers in Handwriting Recognition. Université de Rennes 1, Suvisoft, La Baule, France (2006),,
  5. 5.
    Elagouni, K., Garcia, C., Mamalet, F., Sébillot, P.: Text Recognition in Videos using a Recurrent Connectionist Approach. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 2012, Part II. LNCS, vol. 7553, pp. 172–179. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Gers, F.A., Schraudolph, N.N., Schmidhuber, J.: Learning precise timing with lstm recurrent networks. J. Mach. Learn. Res. 3, 115–143 (2003)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Graves, A.: Offline Arabic Handwriting Recognition with Multidimensional Neural Networks. Springer (2012)Google Scholar
  8. 8.
    Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE TPAMI 31(5), 855–868 (2009)CrossRefGoogle Scholar
  9. 9.
    Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Networks (18), 5–6 (2005)Google Scholar
  10. 10.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation (9), 1735–1780 (1997)Google Scholar
  11. 11.
    Hoffman, M., Varcholik, P., La Viola, J.: Breaking the status quo: Improving 3d gesture recognition with spatially convenient input devices. In: Virtual Reality Conference (VR), pp. 59–66 (2010), doi:10.1109/VR.2010.5444813Google Scholar
  12. 12.
    Hofmann, F.G., Heyer, P., Hommel, G.: Velocity profile based recognition of dynamic gestures with discrete hidden markov models. In: Wachsmuth, I., Fröhlich, M. (eds.) GW 1997. LNCS (LNAI), vol. 1371, pp. 81–95. Springer, Heidelberg (1998), CrossRefGoogle Scholar
  13. 13.
    Kallio, S., Kela, J., Mantyjarvi, J.: Online gesture recognition system for mobile interaction. In: Systems, Man and Cybernetics, vol. 3, pp. 2070–2076 (2003)Google Scholar
  14. 14.
    Kela, J., Korpipää, P., Mäntyjärvi, J., Kallio, S., Savino, G., Jozzo, L., Marca, D.: Accelerometer-based gesture control for a design environment. Personal Ubiquitous Comput. 10(5), 285–299 (2006),, doi:10.1007/s00779-005-0033-8CrossRefGoogle Scholar
  15. 15.
    LaViola Jr., J.J., Zeleznik, R.C.: A practical approach for writer-dependent symbol recognition using a writer-independent symbol recognizer. IEEE Trans. PAMI 29(11), 1917–1926 (2007), CrossRefGoogle Scholar
  16. 16.
    Lefebvre, G., Roux, S., Petit, E.: Automatic temporal segmentation method for instrumented gesture, devices and associated terminal. Patent FR2013/51320 (2013)Google Scholar
  17. 17.
    Liu, J., Wang, Z., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uwave: Accelerometer-based personalized gesture recognition and its applications. In: IEEE PerCom, pp. 1–9 (2009)Google Scholar
  18. 18.
    Mantyla, V.M., Mantyjarvi, J., Seppanen, T., Tuulari, E.: Hand gesture recognition of a mobile device user. In: IEEE ICME, vol. 1, pp. 281–284 (2000)Google Scholar
  19. 19.
    Mayer, H., Gomez, F., Wierstra, D., Nagy, I., Knoll, A., Schmidhuber, J.: A system for robotic heart surgery that learns to tie knots using recurrent neural networks. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (2006)Google Scholar
  20. 20.
    Petit, E.: Grasp: Moteur de reconnaissance de gestes. Inter Desposit Digital Number IDDNFR.001.030023.000.S.P.2010.000.31500 (2010)Google Scholar
  21. 21.
    Pylvänäinen, T.: Accelerometer Based Gesture Recognition Using Continuous HMMs. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3522, pp. 639–646. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  22. 22.
    Wilson, D.H., Wilson, A.: Gesture recognition using the xwand. Tech. Rep. CMU-RI-TR-04-57, Robotics Institute (2004)Google Scholar
  23. 23.
    Woodman, O.J.: An introduction to inertial navigation. Tech. rep., University of Cambridge (2007),
  24. 24.
    Wu, J., Pan, G., Zhang, D., Qi, G., Li, S.: Gesture recognition with a 3-d acceleromete. In: Zhang, D., Portmann, M., Tan, A.-H., Indulska, J. (eds.) UIC 2009. LNCS, vol. 5585, pp. 25–38. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Grégoire Lefebvre
    • 1
    Email author
  • Samuel Berlemont
    • 1
  • Franck Mamalet
    • 2
  • Christophe Garcia
    • 3
  1. 1.Orange LabsR&DMeylanFrance
  2. 2.Orange LabsR&DRennesFrance
  3. 3.LIRIS, UMR 5205 CNRSINSA-LyonFrance

Personalised recommendations