Gesture Aided Voice for Voiceless

  • Dipti Jadhav
  • Tejaswini Koilakuntla
  • Sonam M. Mutalik Desai
  • Ramya Ramakrishnan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11278)


Language is used to express to our thoughts and to communicate with others. It is not limited to speech and sound. A sign language uses gestures which are a combination of hand movements and shapes, orientation, body movements and facial expressions instead of using sound. Sign language is the primary communication medium for the differently abled who have been given the title of “Divyang”. But the inability of common people to comprehend it poses as a problem for efficient communication between the specially abled and others. The idea is to solve the problem by building an wireless and efficient two way communication system to bridge the gap between sign and speech and also vice versa. The system makes use of gloves fitted with flex sensors along the fingers and wrists of both hands to record the bend and corresponding change in resistance while performing Indian Sign Language (ISL) gestures. Microcontroller would be used for mapping the voltage values and its corresponding identifier i.e. the word as it is fast and is effective in a real-time communication scenario. This is then sent to a mobile app using a Bluetooth module where the text is displayed and converted to speech which the normal person can understand. The proposed system would help the hearing and speech impaired community in their daily lives as well as to enhance their employability.


Human machine interaction Indian Sign Language Hand gesture interpretation Microcontroller Natural language processing 


  1. 1.
    Tripathy, A.K., Jadhav, D., Barreto, S.A., Rasquinha, D., Mathew, S.S.: Voice for the mute. In: International Conference on Technologies for Sustainable Development (ICTSD), Mumbai, pp. 1–6 (2015)Google Scholar
  2. 2.
    Bajpai, D., Porov, U., Srivastav, G., Sachan, N.: Two way wireless data communication and american sign language translator glove for images text and speech display on mobile phone. In: Fifth International Conference on Communication Systems and Network Technologies (CSNT), Gwalior, pp. 578–585 (2015)Google Scholar
  3. 3.
    Praveen, N., Karanth, N., Megha, M.S.: Sign language interpreter using a smart glove. In: International Conference on Advances in Electronics, Computers and Communications (ICAECC), Bangalore, pp. 1–5 (2014)Google Scholar
  4. 4.
    Ahire, P.G., Tilekar, K.B., Jawake, T.A., Warale, P.B.: Two way communicator between deaf and dumb people and normal people. In: International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, pp. 641–644 (2015)Google Scholar
  5. 5.
    Chouhan, T., Panse, A., Voona, A.K., Sameer, S.M.: Smart glove with gesture recognition ability for the hearing and speech impaired. In: IEEE 2014 Global Humanitarian Technology Conference - South Asia Satellite (GHTC-SAS), Trivandrum, pp. 105–110 (2014)Google Scholar
  6. 6.
    Gałka, J., Mąsior, M., Zaborski, M., Barczewska, K.: Inertial motion sensing glove for sign language gesture acquisition and recognition. IEEE Sens. J. 16(16), 6310–6316 (2016)CrossRefGoogle Scholar
  7. 7.
    Dekate, A., Kamal, A., Surekha, K.S.: Magic glove - wireless hand gesture hardware controller. In: International Conference on Electronics and Communication Systems (ICECS), Coimbatore, pp. 1–4 (2014)Google Scholar
  8. 8.
    Suresh, P., Vasudevan, N., Ananthanarayanan, N.: Computer-aided interpreter for hearing and speech impaired. In: Fourth International Conference on Computational Intelligence Communication Systems and Networks (CICSyN), Phuket, pp. 248–253 (2012)Google Scholar
  9. 9.
    Meeravali, S., Aparna, M.: Design and development of a hand-glove controlled wheel chair based on MEMS. Int. J. Eng. Trends Technol. (IJETT) 4(8), 3706–3712 (2013). Fig 5: Gestures recognitionGoogle Scholar
  10. 10.
    Patel, B., Shah, V., Kshirsagar, R.: Microcontroller based gesture recognition system for the handicap people. J. Eng. Res. Stud. (JERS) II(IV), 113–115 (2011)Google Scholar
  11. 11.
    Rajalakshmi, V., Vasudevan, N., Rajinigrinath, D., Kumar, S.P.: Electronic hand glove gesture to voice recognization using physically challenged persons. Int. J. Innovative Res. Comput. Commun. Eng 3(8), 90–94 (2015)Google Scholar
  12. 12.
    Singh, S., Tripathi, A.: Gesture recognition using wrist watch. Int. J. Sci. Eng. Res. 4(6), 984–986 (2013)Google Scholar
  13. 13.
    Tripathi, K., Baranwal, N., Nandi, G.C.: Continuous indian sign language gesture recognition and sentence formation. In: Eleventh International Multi-Conference on Information Processing-2015 (IMCIP- 2015), Procedia Computer Science, vol. 54, pp. 523–531 (2015)CrossRefGoogle Scholar
  14. 14.
    Jadhav, D., Tripathy, A.K., Jose, C., Fernandes, R., Joy, J.: Gesture aided speech-framework, unpublishedGoogle Scholar
  15. 15.
    Cho, I.-Y., Sunwoo, J., Son, Y.-K., Oh, M.-H., Lee, C.-H.: Development of a Single 3-axis Accelerometer Sensor Based Wearable Gesture Recognition Band, unpublished Google Scholar
  16. 16.
  17. 17.
    Sign language Introduction. Accessed 30 Oct
  18. 18.
  19. 19.
  20. 20.
  21. 21.

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Dipti Jadhav
    • 1
  • Tejaswini Koilakuntla
    • 1
  • Sonam M. Mutalik Desai
    • 1
  • Ramya Ramakrishnan
    • 1
  1. 1.Department of Computer EngineeringDon Bosco Institute of TechnologyMumbaiIndia

Personalised recommendations