A Generic Multi-modal Dynamic Gesture Recognition System Using Machine Learning

  • G. Gautham KrishnaEmail author
  • Karthik Subramanian Nathan
  • B. Yogesh Kumar
  • Ankith A. Prabhu
  • Ajay Kannan
  • Vineeth Vijayaraghavan
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 887)


Human computer interaction facilitates intelligent communication between humans and computers, in which gesture recognition plays a prominent role. This paper proposes a machine learning system to identify dynamic gestures using tri-axial acceleration data acquired from two public datasets. These datasets, uWave and Sony, were acquired using accelerometers embedded in Wii remotes and smartwatches, respectively. A dynamic gesture signed by the user is characterized by a generic set of features extracted across time and frequency domains. The system was analyzed from an end-user perspective and was modelled to operate in three modes. The modes of operation determine the subsets of data to be used for training and testing the system. From an initial set of seven classifiers, three were chosen to evaluate each dataset across all modes rendering the system towards mode-neutrality and dataset-independence. The proposed system is able to classify gestures performed at varying speeds with minimum preprocessing, making it computationally efficient. Moreover, this system was found to run on a low-cost embedded platform – Raspberry Pi Zero (USD 5), making it economically viable.


Gesture recognition Accelerometers Feature extraction Machine learning algorithms 



The authors would like to thank Solarillion Foundation for its support and funding of the research work carried out.


  1. 1.
    Chen, Q., Georganas, N.D., Petriu, E.M.: Real-time vision-based hand gesture recognition using HAAR-like features. In: 2007 IEEE Instrumentation & Measurement Technology Conference (IMTC 2007), Warsaw, pp. 1–6 (2007)Google Scholar
  2. 2.
    Biswas, K.K., Basu, S.K.: Gesture recognition using Microsoft Kinect. In: The 5th International Conference on Automation, Robotics and Applications, Wellington, pp. 100–103 (2011)Google Scholar
  3. 3.
    Mitra, S., Acharya, T.: Gesture recognition: a survey. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 37(3), 311–324 (2007)CrossRefGoogle Scholar
  4. 4.
    Liu, J., Wang, Z., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uWave: accelerometer-based personalized gesture recognition and its applications. In: 2009 IEEE International Conference on Pervasive Computing and Communications, Galveston, TX, pp. 1–9 (2009)Google Scholar
  5. 5.
    SmartWatch Gestures Dataset, Technologies of Vision, Fondazione Bruno Kessler.
  6. 6.
    Zimmerman, T.G., Lanier, J., Blanchard, C., Bryson, S., Harvill, Y.: Hand gesture interface device. In: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, pp. 189–192 (1986)Google Scholar
  7. 7.
    Hussain, S.M.A., Rashid, A.B.M.H.: User independent hand gesture recognition by accelerated DTW. In: 2012 International Conference on Informatics, Electronics & Vision (ICIEV), Dhaka, pp. 1033–1037 (2012)Google Scholar
  8. 8.
    Pylvänäinen, T.: Accelerometer based gesture recognition using continuous HMMs. In: IbPRIA, vol. 3522, pp. 639–646 (2005). 492Google Scholar
  9. 9.
    Helmi, N., Helmi, M.: Applying a neuro-fuzzy classifier for gesture-based control using a single wrist-mounted accelerometer. In: IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA), Daejeon, pp. 216–221 (2009)Google Scholar
  10. 10.
    Altun, K., Barshan, B., Tunçel, O.: Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognit. 43(10), 3605–3620 (2010)CrossRefGoogle Scholar
  11. 11.
    Jing, L., Zhou, Y., Cheng, Z., Wang, J.: A recognition method for one-stroke finger gestures using a MEMS 3D accelerometer. IEICE Trans. 94-D(5), 1062–1072 (2011)CrossRefGoogle Scholar
  12. 12.
    Wu, J., Pan, G., Zhang, D., Qi, G., Li, S.: Gesture recognition with a 3-D accelerometer. In: UIC, vol. 5585, pp. 25–38 (2009)CrossRefGoogle Scholar
  13. 13.
    Ishida, H., Takahashi, T., Ide, I., Murase, H.: A Hilbert warping method for handwriting gesture recognition. Pattern Recognit. 43(8), 2799–2806 (2010)CrossRefGoogle Scholar
  14. 14.
    Tencer, L., Reznáková, M., Cheriet, M.: Evaluation of techniques for signature classification from accelerometer and gyroscope data. In: ICDAR, pp. 1066–1070 (2015)Google Scholar
  15. 15.
    Aswolinskiy, W., Reinhart, R.F., Steil, J.J.: Impact of regularization on the model space for time series classification. In: Machine Learning Reports, pp. 49–56 (2015)Google Scholar
  16. 16.
    Yang, Y., Yu, Y.: A hand gestures recognition approach combined attribute bagging with symmetrical uncertainty. In: 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, Sichuan, pp. 2551–2554 (2012)Google Scholar
  17. 17.
    Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • G. Gautham Krishna
    • 1
    Email author
  • Karthik Subramanian Nathan
    • 2
  • B. Yogesh Kumar
    • 1
  • Ankith A. Prabhu
    • 3
  • Ajay Kannan
    • 2
  • Vineeth Vijayaraghavan
    • 1
  1. 1.Research and Outreach, Solarillion FoundationChennaiIndia
  2. 2.College of Engineering, GuindyChennaiIndia
  3. 3.SRM UniversityChennaiIndia

Personalised recommendations