Hand Gesture Recognition Based Omnidirectional Wheelchair Control Using IMU and EMG Sensors

  • Ananda Sankar Kundu
  • Oishee Mazumder
  • Prasanna Kumar Lenka
  • Subhasis Bhaumik


This paper presents a hand gesture based control of an omnidirectional wheelchair using inertial measurement unit (IMU) and myoelectric units as wearable sensors. Seven common gestures are recognized and classified using shape based feature extraction and Dendogram Support Vector Machine (DSVM) classifier. The dynamic gestures are mapped to the omnidirectional motion commands to navigate the wheelchair. A single IMU is used to measure the wrist tilt angle and acceleration in three axis. EMG signals are extracted from two forearm muscles namely Extensor Carpi Radialis and Flexor Carpi Radialis and processed to provide Root Mean Square (RMS) signal. Initiation and termination of dynamic activities are based on autonomous identification of static to dynamic or dynamic to static transition by setting static thresholds on processed IMU and myoelectric sensor data. Classification involves recognizing the activity pattern based on periodic shape of trajectories of the triaxial wrist tilt angle and EMG-RMS from the two selected muscles. Second order Polynomial coefficients extracted from the sensor trajectory templates during specific dynamic activity cycles are used as features to classify dynamic activities. Classification algorithm and real time navigation of the wheelchair using the proposed algorithm has been tested by five healthy subjects. Classification accuracy of 94% was achieved by DSVM classifier on ‘k’ fold cross validation data of 5 users. Classification accuracy while operating the wheelchair was 90.5%.


Dendogram support vector machine Myoelectric Inertial measurement unit Omni wheel Holonomic control 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Madarasz, R.L., Heiny, L.C., Cromp, R.F., Mazur, N.M.: The design of an autonomous vehicle for the disabled. IEEE J. Robot. Autom. 2(3), 117–126 (1986)CrossRefGoogle Scholar
  2. 2.
    Simpson, R.C.: Smart wheelchair: a literature review. J. Rehab. Res. Develop. 42(4), 423–436 (2005)CrossRefGoogle Scholar
  3. 3.
    Krishnan, R.H., Pugazhenthi, S.: Mobility assistive devices and self-transfer robotic systems for elderly, a review. Intell. Service Robot. 7, 37–49 (2014)CrossRefGoogle Scholar
  4. 4.
    Levine, S.P., Bell, D.A., Jaros, L.A., Simpson, R.C., Koren, Y., Borenstein, J.: The NavChair assistive wheelchair navigation system. IEEE Trans. Rehab. Eng. 7(4), 443–51 (1999)CrossRefGoogle Scholar
  5. 5.
    Borgolte, U., Hoyer, H., Buehler, C., Heck, H., Hoelper, R.: Architectural concepts of a semi-autonomous wheelchair. J. Intell. Robot. Syst. 22(3), 233–53 (1998)CrossRefGoogle Scholar
  6. 6.
    Prassler, E., Scholz, J., Fiorini, P.: A robotic wheelchair for crowded public environments. IEEE Robot. Autom. Mag. 8(1), 38–45 (2001)CrossRefGoogle Scholar
  7. 7.
    Simpson, R.C., LoPresti, E.F., Hayashi, S., Guo, S., Ding, D., Cooper, R.A.: Smart power assistance module for manual wheelchairs. technology and disability: research, design, practice and policy. In: 26th International Annual Conference on Assistive Technology for People with Disabilities (RESNA), 2003 Jun 19–23; Atlanta, GA. RESNA Press, Arlington (2003)Google Scholar
  8. 8.
    Miller, D.P., Slack, M.G.: Design and testing of a low-cost robotic wheelchair prototype. Auton. Robot. 2 (1), 77–88 (1995)CrossRefGoogle Scholar
  9. 9.
    Fehr, L., Langbein, W.E., Skaar, S.B.: Adequacy of power wheelchair control interfaces for persons with severe disabilities: a clinical survey. J. Rehabil. Res. Dev. 37(3), 353–60 (2000)Google Scholar
  10. 10.
    Jia, P., Hu, H., Lu, T., Yuan, K.: Head gesture recognition for hand-free control of an intelligent wheelchair. Int. J. Ind. Robot. 34(1), 60–8 (2007)CrossRefGoogle Scholar
  11. 11.
    Nakanishi, S., Kuno, Y., Shimada, N., Shirai, Y.: Robotic wheelchair based on observations of both user and environment. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, pp. 912–7 (1999)Google Scholar
  12. 12.
    Katevas, N.I., Sgouros, N.M., Tzafestas, S.G., Papakonstantinou, G., Beattie, P., Bishop, J.M.: The autonomous mobile robot SENARIO: a sensor-aided intelligent navigation system for powered wheelchairs. IEEE Robot. Autom. Mag. 4(4), 60–70 (1997)CrossRefGoogle Scholar
  13. 13.
    Barea, R., Boquete, L., Mazo, M., Lopes, E.: System for assisted mobility using eye movements based on electrooculography. IEEE Trans. Neural Syst. Rehabil. Eng. 10(4), 209–18 (2002)CrossRefGoogle Scholar
  14. 14.
    Tanaka, K., Matsunaga, K., Wang, H.O.: Electroencephalogram based control of an electric wheelchair. IEEE Trans. Robot. 21(4), 762–6 (2005)CrossRefGoogle Scholar
  15. 15.
    Oskoei, M.A., Hu, H.: Myoelectric control systems - a survey. Biomed. Signal Process. Control 2(4), 275–94 (2007)CrossRefGoogle Scholar
  16. 16.
    Ahsan, M.R., Ibrahimy, M.I., Khalifa, O.O.: Advances in electro- myogram signal classification to improve the quality of life for the disabled and aged people. J. Comput. Sci. 7(6), 706–715 (2010)CrossRefGoogle Scholar
  17. 17.
    Khezri, M., Jahed, M.: A novel approach to recognize hand movements via semg patterns. In: 29th Annual International Conference of the IEEE EMBS, pp. 4907–4910 (2007)Google Scholar
  18. 18.
    Shuman, G.: Using forearm electromyograms to classify hand gestures. In: IEEE International Conference on Bioinformatics and Biomedicine, pp. 261–264 (2009)Google Scholar
  19. 19.
    Saponas, T.S., Tan, D.S., Morris, D., Balakrishnan, R.: Demonstrating the feasibility of using forearm electromyography for muscle–computer interfaces. In: Proceedings of 26th SIGCHI Conference Human Factors Computing System, Florence, Italy, pp. 515–524 (2008)Google Scholar
  20. 20.
    Wheeler, K.R., Chang, M.H., Knuth, K.H.: Gesture-based control and EMG decomposition. IEEE Trans. Syst., Man, Cybern. C, Appl. Rev. 36(4), 503–514 (2006)CrossRefGoogle Scholar
  21. 21.
    Sherrill, D.M., Bonato, P., De Luca, C.J.: A neural network approach to monitor motor activities. In: Proceedings of 2nd Joint EMBS/BMES Conference, Houston, TX, vol. 1, pp. 52–53 (2002)Google Scholar
  22. 22.
    Zhang, X., Chen, X., Li, Y., Lantz, V., Wang, K., Yang, J.: A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans. Syst. Man Cybern. Syst. Hum. 41(6), 1064–1076 (2011)CrossRefGoogle Scholar
  23. 23.
    Lajnefa, T., Chaibia, S., Rubyb, P., Aguerab, P.-E., Eichenlaubc, J.-B., Sameta, M., Kachouria, A., Jerbib, K.: Learning machines and sleeping brains: automatic sleep stage classification using decision-tree multi-class support vector machines. J. Neurosci. Methods 30(250), 94–105 (2015)CrossRefGoogle Scholar
  24. 24.
    Benabdeslem, K., Bennani, Y.: Dendogram-based SVM for multi-class classification. J. Comput. Inf. Technol. - CIT 14 4, 283–289 (2006).  https://doi.org/10.2498/cit.2006.04.03 CrossRefGoogle Scholar
  25. 25.
    Allen, F.R., Ambikairajah, E., Lovell, N.H., Celler, B.G.: Classification of a known sequence of motions and postures from accelerometry data using adapted Gaussian mixture models. Physiol. Meas. 27, 935–951 (2006)CrossRefGoogle Scholar
  26. 26.
    Ermes, M., Parkka, J., Mantyjarvi, J., Korhonen, I.: Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions. IEEE Trans. Inf. Technol. Biomed. 12(1), 20–26 (2008)CrossRefGoogle Scholar
  27. 27.
    Olguin, D.O., Pentland, A.: Human activity recognition: accuracy across common locations for wearable sensors. In: 10th International Symposium of Wearable Computer (Student Colloquium), Montreux, Switzerland (2006)Google Scholar
  28. 28.
    Watanabe, K., Shiraishi, Y., Tzafestas, S.G., Tang, J., Fukuda, T.: Feedback control of an omnidirectional autonomous platform for mobile service robots. J. Intell. Robot. Syst. 22(3), 315–330 (1998)CrossRefGoogle Scholar
  29. 29.
    Conceic ao, A.S., Moreira, A.P., Costa, P.J.: Model identification of a four wheeled omni-directional mobile robot. In: Controlo 2006, 7th Portuguese Conference on Automatic Control, Instituto Superior Tecnico, Lisboa, Portugal (2006)Google Scholar
  30. 30.
    Lajnef, T.: Multiclass SVM classifier. http://www.mathworks.com/matlabcentral/fileexchange/48632-multiclass-svm-classifier. Accessed 12th August, 2016

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  • Ananda Sankar Kundu
    • 1
  • Oishee Mazumder
    • 1
  • Prasanna Kumar Lenka
    • 2
  • Subhasis Bhaumik
    • 1
  1. 1.IIESTShibpurIndia
  2. 2.NIOHKolkataIndia

Personalised recommendations