Abstract
For deafblind individuals, the absence of visual and auditory communication channels prevents meaningful interactions with the people and world around them, leading many to suffer from both mental and social issues. Among the remaining sensory channels (e.g., taste, haptic, and olfactory) of deafblind individuals, haptic stimuli are an effective medium that can be digitally employed to enable interactions with the outside world. Thus, this paper discusses the development of a digital communication platform for deafblind individuals called “EnPower" (Enable and Empower individuals with auditory and visual impairments). EnPower enables bi-directional communication between deafblind and non-impaired individuals by using sensory substitution techniques to present information via accessible haptic stimuli. The EnPower platform consists of a physical interface (in wearable and desktop versions) that is paired with a wireless mobile application. Similar to concepts such as Finger Braille, the assistive devices can deliver textual information via tactile stimuli to the deafblind individual, and vice versa. Additionally, the system can translate speech and visual input into tactile stimuli, providing users with greater access to digital information platforms. Using the two devices, a stimuli perception study, as well as a field trial, was conducted to evaluate the effectiveness of our approach (delivering information via tactile sensations on fingertips). Findings not only suggest that participants could understand the information presented via tactile sensations but also reveal several important avenues for future research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bach-y Rita, P., Kercel, S.W.: Sensory substitution and the human–machine interface. Trends Cogn. Sci. 7(12), 541–546 (2003)
Fukumoto, M., Tonomura, Y.: “Body coupled FingerRing" : wireless wearable keyboard. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 147–154. ACM (1997)
Fukushima, S.: Person with DeafBlind and Normalization. Akashi Shoten, Tokyo (1997)
Gollner, U., Bieling, T., Joost, G.: Mobile lorm glove: introducing a communication device for deaf-blind people. In: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, pp. 127–130. ACM (2012)
Hoshino, T., Otake, T., Yonezawa, Y.: A study on a finger-braille input system based on acceleration of finger movements. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. 85(3), 731 (2002)
Koo, I.M., Jung, K., Koo, J.C., Nam, J.-D., Lee, Y.K., Choi, H.R.: Development of soft-actuator-based wearable tactile display. IEEE Trans. Robot. 24(3), 549–558 (2008)
Lee, J., Kim, Y., Kim, G.: Funneling and saltation effects for tactile interaction with virtual objects. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’2012, New York, NY, USA, pp. 3141–3148. ACM (2012)
Ochi, T., Kozuki, T., Suga, H.: Bracelet type braille interface. Correspondence Hum. Interface, 5(1) (2003)
Osada, K., Katayama, S., Wang, K., Kitajima, K.: Development of communication system with finger braille robot for deaf-blind people. Proc. Hum. Interface Symp. 2000, 41–44 (2000)
Peiris, R.L., Feng, Y.-L., Chan, L., Minamizawa, K.: Thermalbracelet: exploring thermal haptic feedback around the wrist. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI’2019, New York, NY, USA, pp. 170:1–170:11. ACM (2019)
Ramirez-Garibay, F., Olivarria, C.M., Aguilera, A.F.E., Huegel, J.C.: Myvox—device for the communication between people: blind, deaf, deaf-blind and unimpaired. In: IEEE Global Humanitarian Technology Conference (GHTC 2014), pp. 506–509. IEEE (2014)
Ranasinghe, N., Jain, P., Karwita, S., Tolley, D., Do, E.Y.-L.: Ambiotherm: enhancing sense of presence in virtual reality by simulating real-world environmental conditions. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI’2017, New York, NY, USA, pp. 1731–1742. ACM (2017)
Ranasinghe, N., Jain, P., Tram, N.T.G., Koh, K.C.R., Tolley, D., Karwita, S., Liangkun, L.-Y.Y., Shamaiah, K., Tung, C.E.W., Yen, C.C., Do, E.Y.-L.: Season traveller: multisensory narration for enhancing the virtual reality experience. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI’2018, New York, NY, USA, pp. 577:1–577:13. ACM (2018)
Ranasinghe, N., Tram Nguyen, T.N., Liangkun, Y., Lin, L.-Y., Tolley, D., Do, E.Y.-L.: Vocktail: a virtual cocktail for pairing digital taste, smell, and color sensations. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1139–1147 (2017)
Ranasinghe, N., Tolley, D., Tram Nguyen, T.N., Yan, L., Chew, B., Do, E.Y.-L., Modulation of flavour experiences through electric taste augmentation: Augmented flavours. Food Res. Int. 117, 60–68 (2019)
Robertson, J., Emerson, E.: Estimating the Number of People with Co-Occurring Vision and Hearing Impairments in the UK. Centre for Disability Research, Lancaster University, Lancaster (2010)
Tewell, J., Bird, J., Buchanan, G.R.: Heat-NAV: using temperature changes as navigation cues. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI’2017, New York, NY, USA, pp. 1131–1135. ACM (2017)
Tolley, D., Tram Nguyen, T.N., Tang, A., Ranasinghe, N., Kawauchi, K., Yen, C.C.: Windywall: exploring creative wind simulations. In: Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI’2019, New York, NY, USA, pp. 635–644. ACM (2019)
Turner, O., Windfuhr, K., Kapur, N.: Suicide in deaf populations: a literature review. Ann. Gen. Psychiatry 6(1), 26 (2007)
Uehara, N., Aoki, M., Nagashima, Y., Miyoshi, K.: Bidirectional telecommunication using wearable i/o terminal for finger braille. Proc. Hum. Interface Symp. 2000, 2000–9 (2000)
Velázquez, R.: Wearable assistive devices for the blind. In: Wearable and Autonomous Biomedical Devices and Systems for Smart Environment, pp. 331–349. Springer (2010)
Acknowledgments
This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centres in Singapore Funding Initiative.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ranasinghe, N. et al. (2021). EnPower: Haptic Interfaces for Deafblind Individuals to Interact, Communicate, and Entertain. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Proceedings of the Future Technologies Conference (FTC) 2020, Volume 2 . FTC 2020. Advances in Intelligent Systems and Computing, vol 1289. Springer, Cham. https://doi.org/10.1007/978-3-030-63089-8_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-63089-8_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63088-1
Online ISBN: 978-3-030-63089-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)