Abstract
According to the census of 2011, about 10.6 million people are visually impaired, 1.2 million people have hearing disability, and 1.6 million people are speech impaired in India. In this paper, we propose an Internet of Thing (IoT)-based integrated device for these people in order to provide them a user-friendly and interactive environment through which they can easily communicate with others and interact with their surroundings. The device provides advanced features like facial recognition, emotion recognition, object recognition, sign language recognition, cloth pattern and color recognition, distance from an object, and optical character recognition. By giving input in the form of voice, text, and sign, one can use different functionalities of the device. Algorithms like image-to-text, text-to-speech, and speech-to-text conversion are implemented to provide intended functionality. Convolutional Neural Network (CNN) is used for feature extraction and classification. The proposed device considers various communications between speech, vision, and hearing impaired people with others.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kavita and Manjeet Kaur, A survey paper for face recognition technologies. International journal for scientific and research publications 6(7), 441–445 (2016)
N. Roopa, Emotion recognition from facial expression using deep learning. International journal of engineering and advanced technology 8(6), 91–95 (2019)
Monika Dubey and Prof. Lokesh Singh. Automatic emotion recognition using facial expression: A review. International research journal of engineering and technology, Volume 3, issue 2, pp 488–492 (2016).
Dilbag Singh. Human Emotion recognition system. I.J. Image, Graphics and Signal Processing, 8, 50–56, 14, pp 50–56 (2012).
Jiwoong Choi, Dayoung Chun Hyun Kim and Hyuk-Jae Lee. Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving. IEEE, pp 502–511.
A. Vidyavani , K. Dheeraj, M. Rama Mohan Reddy, KH. Naveen Kumar. Object Detection Method Based on YOLOv3 using Deep Learning Networks.International Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN: 2278–3075, Volume-9 Issue- 1, pp 1414–1417 (2019).
S. Kumar, A. Balyan, M. Chawla, Object Detection and Recognition in Images. International Journal of Engineering Development and Research 5(4), 1029–1034 (2017)
Juan Wu1, Bo Peng1, Zhenxiang Huang1, and Jietao Xie2Research on Computer Vision-Based Object Detection and Classification, pp 183–188.
Joyeeta Singha and Karen Das Indian Sign Language Recognition Using Eigen Value Weighted Euclidean Distance Based Classification Technique. International Journal of Advanced Computer Science and Applications, Vol. 4, No. 2, pp 188–195 (2013).
Pradip Patel And Narendra Patel, Vision Based Real-time Recognition of Hand Gestures for Indian Sign Language using Histogram of Oriented Gradients Features. International Journal of Next-Generation Computing 10(2), 92–102 (2019)
Debesh Chaudhary and Sunitha Beevi K.Spotting and Recognition of Hand Gesture for Indian Sign Language using Skin Segmentation with YCbCr and HSV Color Models under different Lighting Conditions. International Journal of Innovations & Advancement in Computer Science Volume 6, Issue 9, pp 426–435 (2017).
Sachin Bhat, Amruthesh M, Ashik, Chidanand Das and Sujith. Translating Indian Sign Language to text andvoice messages using flex sensors. International Journal of Advanced Research in Computer and Communication Engineering Vol. 4, Issue 5, pp 430–434 (2015).
Mrs. Anuradha.S.G, Thogaricheti Ashwini.Clothing Color and Pattern Recognition for Impaired people. International Journal Of Engineering And Computer Science ISSN: 2319–7242 Volume 5, Issue 4, pp 16317–16324 (2016).
Yannis Kalantidis, Lyndon Kennedy and Li-Jia Li. Getting the Look: Clothing Recognition and Segmentation for Automatic Product Suggestions in Everyday Photos. Conference Paper (2013).
H.M. Jawad, T.A. Husain, Measuring Object Dimensions and its Distances Based on Image Processing Technique by Analysis the Image Using Sony Camera. Eurasian Journal of Science & Engineering 3(2), 100–110 (2017)
A.K. Shrivastava, A. Verma, S.P. Singh, Distance Measurement of an Object or Obstacle by Ultrasound Sensors using P89C51RD2. International Journal of Computer Theory and Engineering 2(1), 64–68 (2010)
Vedasamhitha Abburu, Saumya Gupta, S. R. Rimitha, Manjunath Mulimani, Shashidhar G. Koolagudi.Currency recognition system using image processing. Proceedings of 2017 Tenth International Conference on Contemporary Computing (IC3), Noida, India (2017).
Kiran Swami, Kshitija Murumkar, Dr. Premanand Ghadekar, Yash Solanki.Currency Recognition and fake currency identification using image processing.International Journal of Advance Research, Ideas and Innovation in Technology.Volume 5, issue 3, pp 659–661 (2019).
Muhammad Sarfraz.An intelligent paper currency recognition system. International Conference on Communication, Management and Information Technology, pp 538–545 (2015).
Karmel A, Anushka Sharma, Muktak Pandya, Diksha Garg. IoT based assistive device for deaf, dumb and blind people. International conference on recent trends in advanced computing, pp 259–269 (2019)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Umraniya, C., Timbal, M., Tandel, K., Prajapati, D., Patel, P. (2022). Communication Assistant Using IoT-Based Device for People with Vision, Speech, and Hearing Disability. In: Luhach, A.K., Poonia, R.C., Gao, XZ., Singh Jat, D. (eds) Second International Conference on Sustainable Technologies for Computational Intelligence. Advances in Intelligent Systems and Computing, vol 1235. Springer, Singapore. https://doi.org/10.1007/978-981-16-4641-6_2
Download citation
DOI: https://doi.org/10.1007/978-981-16-4641-6_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-4640-9
Online ISBN: 978-981-16-4641-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)