Skip to main content
SpringerLink
Log in

Transforming consulting atmosphere with Indian sign language translation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

One of the major challenges faced by medical practitioners is to facilitate hassle free communication with speech impaired people. It is difficult for speech impaired people to communicate their illness to a physician as most physicians do not understand sign language. Sign languages are languages that use the visual-manual modality to convey meaning. Sign languages are expressed through manual articulations in combination with non-manual elements. Indian Sign Language (ISL) is the most widely used sign language by the deaf and hearing-impaired community in India. Hearing impaired people face challenges during consultation with doctors in communicating the symptoms effectively due to the inability of doctors to understand sign language. The situation becomes even worse when the consultation is through an online platform. The existing models recognise hand gestures from videos, compare the recognised hand spatial orientation with the annotated dataset, and identify the letters or words. The drawbacks of the existing systems are, they neither include the entire vocabulary in the dataset nor are they contextual based. Hence these models are not very effective in a real-life communication situation. The datasets available are limited to recognition of alphabets, digits, some words like greeting terms, bank transaction terms etc. There has been little or no work done related to recognition of medical symptoms from Indian Sign Language. The proposed model will recognise patients’ hand gestures in the video captured and translate it to corresponding words. Short sentences will be framed from the words recognised, in order to ensure clear communication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

The datasets generated during the current study are available in the repository https://github.com/snehakumares/Sign_recognition_dataset.git.

References

  1. Al-Hammadi M, Muhammad G, Abdul W, Alsulaiman M, Bencherif MA, Mekhtiche MA (2020) Hand Gesture Recognition for Sign Language Using 3DCNN. IEEE Access 8:79491–79509

    Article  Google Scholar 

  2. Amritkar, C, Jabade V (2018) Image caption generation using deep learning technique. In 2018 fourth international conference on computing communication control and automation (ICCUBEA), pp. 1–4. IEEE

  3. Basnin N, Nahar L, Hossain MS (2021) An Integrated Model for Bangla Lexical Sign Language Recognition In Proceedings of International Conference on Trends in Computational and Cognitive Engineering, pp. 695–707. Springer, Singapore

  4. Chandra MM, Rajkumar S, Kumar LS (2019) Sign languages to speech conversion prototype using the SVM classifier. TENCON 2019 - 2019 IEEE Region 10 Conference (TENCON)

  5. Dignan C, Perez E, Ahmad I, Huber M, Clark A (2022) An AI-based Approach for Improved Sign Language Recognition using Multiple Videos. Multimed Tools Appl

  6. Gangadia D, Chamaria V, Doshi V, Gandhi J (2020) Indian sign language interpretation and sentence formation. In 2020 IEEE Pune Section International Conference (PuneCon), pp. 71–76. IEEE

  7. Gangrade J, Bharti J (2020) Vision-based hand gesture recognition for Indian sign language using convolution neural network. IETE J Res 1–10

  8. Hatibaruah D, Talukdar AK, Kumar Sarma K (2020) A static hand gesture based sign language recognition system using convolutional neural networks. 2020 IEEE 17th India Council International Conference (INDICON)

  9. Tomasz Kapuscinski, Marian Wysocki (2020) Recognition of signed expressions in an experimental system supporting deaf clients in the city office. Sensors 20(8):2190

    Article  Google Scholar 

  10. Qu Y, Liu P, Song W, Liu L, Cheng M (2020) A text generation and prediction system: pre-training on new corpora using BERT and GPT-2. In 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC), pp. 323–326. IEEE

  11. Sharma S, Kumar K (2021) ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks. Multimed Tools Appl 1–13

  12. Sharma S, Singh S (2021) Vision-based hand gesture recognition using deep learning for the interpretation of sign language. Expert Syst Appl 182:115657

    Article  Google Scholar 

  13. Shrenika S, Madhu Bala M (2020) Sign language recognition using template matching technique. 2020 International Conference on Computer Science, Engineering and Applications (ICCSEA)

  14. Wadhawan A, Kumar P (2020) Deep learning-based sign language recognition system for static signs. Neural Comput Appl 32:7957–7968

    Article  Google Scholar 

  15. Yirtici T, Yurtkan K (2022) Regional-CNN-based enhanced Turkish sign language recognition. Signal, Image Vid Process

Download references

Author information

Authors and Affiliations

  1. B. Tech in Compuer Science, NSS College Of Engineering, Palakkad, Kerala, India

    Haritha V. Das, Kavya Mohan, Linta Paul & Sneha Kumaresan

  2. Department of Computer Science, NSS College of Engineering, Palakkad, Kerala, India

    Chitra S. Nair

Authors
  1. Haritha V. Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kavya Mohan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Linta Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sneha Kumaresan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chitra S. Nair
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haritha V. Das.

Ethics declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Consent to Participate

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Das, H.V., Mohan, K., Paul, L. et al. Transforming consulting atmosphere with Indian sign language translation. Multimed Tools Appl 83, 13543–13555 (2024). https://doi.org/10.1007/s11042-023-15214-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15214-2

Keywords

Search

Navigation