Skip to main content

A Generic Multi-modal Dynamic Gesture Recognition System Using Machine Learning

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 887)


Human computer interaction facilitates intelligent communication between humans and computers, in which gesture recognition plays a prominent role. This paper proposes a machine learning system to identify dynamic gestures using tri-axial acceleration data acquired from two public datasets. These datasets, uWave and Sony, were acquired using accelerometers embedded in Wii remotes and smartwatches, respectively. A dynamic gesture signed by the user is characterized by a generic set of features extracted across time and frequency domains. The system was analyzed from an end-user perspective and was modelled to operate in three modes. The modes of operation determine the subsets of data to be used for training and testing the system. From an initial set of seven classifiers, three were chosen to evaluate each dataset across all modes rendering the system towards mode-neutrality and dataset-independence. The proposed system is able to classify gestures performed at varying speeds with minimum preprocessing, making it computationally efficient. Moreover, this system was found to run on a low-cost embedded platform – Raspberry Pi Zero (USD 5), making it economically viable.


  • Gesture recognition
  • Accelerometers
  • Feature extraction
  • Machine learning algorithms

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Chen, Q., Georganas, N.D., Petriu, E.M.: Real-time vision-based hand gesture recognition using HAAR-like features. In: 2007 IEEE Instrumentation & Measurement Technology Conference (IMTC 2007), Warsaw, pp. 1–6 (2007)

    Google Scholar 

  2. Biswas, K.K., Basu, S.K.: Gesture recognition using Microsoft Kinect. In: The 5th International Conference on Automation, Robotics and Applications, Wellington, pp. 100–103 (2011)

    Google Scholar 

  3. Mitra, S., Acharya, T.: Gesture recognition: a survey. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 37(3), 311–324 (2007)

    CrossRef  Google Scholar 

  4. Liu, J., Wang, Z., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uWave: accelerometer-based personalized gesture recognition and its applications. In: 2009 IEEE International Conference on Pervasive Computing and Communications, Galveston, TX, pp. 1–9 (2009)

    Google Scholar 

  5. SmartWatch Gestures Dataset, Technologies of Vision, Fondazione Bruno Kessler.

  6. Zimmerman, T.G., Lanier, J., Blanchard, C., Bryson, S., Harvill, Y.: Hand gesture interface device. In: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, pp. 189–192 (1986)

    Google Scholar 

  7. Hussain, S.M.A., Rashid, A.B.M.H.: User independent hand gesture recognition by accelerated DTW. In: 2012 International Conference on Informatics, Electronics & Vision (ICIEV), Dhaka, pp. 1033–1037 (2012)

    Google Scholar 

  8. Pylvänäinen, T.: Accelerometer based gesture recognition using continuous HMMs. In: IbPRIA, vol. 3522, pp. 639–646 (2005). 492

    Google Scholar 

  9. Helmi, N., Helmi, M.: Applying a neuro-fuzzy classifier for gesture-based control using a single wrist-mounted accelerometer. In: IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA), Daejeon, pp. 216–221 (2009)

    Google Scholar 

  10. Altun, K., Barshan, B., Tunçel, O.: Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognit. 43(10), 3605–3620 (2010)

    CrossRef  Google Scholar 

  11. Jing, L., Zhou, Y., Cheng, Z., Wang, J.: A recognition method for one-stroke finger gestures using a MEMS 3D accelerometer. IEICE Trans. 94-D(5), 1062–1072 (2011)

    Google Scholar 

  12. Wu, J., Pan, G., Zhang, D., Qi, G., Li, S.: Gesture recognition with a 3-D accelerometer. In: UIC, vol. 5585, pp. 25–38 (2009)

    Google Scholar 

  13. Ishida, H., Takahashi, T., Ide, I., Murase, H.: A Hilbert warping method for handwriting gesture recognition. Pattern Recognit. 43(8), 2799–2806 (2010)

    CrossRef  Google Scholar 

  14. Tencer, L., Reznáková, M., Cheriet, M.: Evaluation of techniques for signature classification from accelerometer and gyroscope data. In: ICDAR, pp. 1066–1070 (2015)

    Google Scholar 

  15. Aswolinskiy, W., Reinhart, R.F., Steil, J.J.: Impact of regularization on the model space for time series classification. In: Machine Learning Reports, pp. 49–56 (2015)

    Google Scholar 

  16. Yang, Y., Yu, Y.: A hand gestures recognition approach combined attribute bagging with symmetrical uncertainty. In: 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, Sichuan, pp. 2551–2554 (2012)

    Google Scholar 

  17. Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)

    CrossRef  Google Scholar 

Download references


The authors would like to thank Solarillion Foundation for its support and funding of the research work carried out.

Author information

Authors and Affiliations


Corresponding author

Correspondence to G. Gautham Krishna .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Krishna, G.G., Nathan, K.S., Kumar, B.Y., Prabhu, A.A., Kannan, A., Vijayaraghavan, V. (2019). A Generic Multi-modal Dynamic Gesture Recognition System Using Machine Learning. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Advances in Information and Communication Networks. FICC 2018. Advances in Intelligent Systems and Computing, vol 887. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03404-7

  • Online ISBN: 978-3-030-03405-4

  • eBook Packages: EngineeringEngineering (R0)