Augmentative and Alternative Interaction Service with AI Speakers to Access Home IoT Devices and Internet Services for People with Multiple Disabilities

Abstract

With the growing demand for Internet-of-Things (IoT) and internet services for people with multiple disabilities, including communication and motor disabilities, we propose an Augmentative and Alternative Interaction (AAI) service for them. Interacting with IoT devices and internet services can be done by voice-controllable smart (or Artificial Intelligence (AI)) speakers. AI speakers are difficult to use for those people with multiple disabilities. The direct application of Augmentative and Alternative Communication (AAC) to the interaction is ineffective, since AAC mainly focuses on face-to-face communication and the sound produced by Text-to-Speech (TTS) for AAC symbols does not reflect the AI speakers’ speech recognition features. Based on a user survey for AI speaker commands, we developed AAI symbols and boards to interact with home IoT devices and internet services through AI speakers. In order to improve speech recognition performance of AI speakers, we developed the TTS production format for AAI service, including wake-up words, silence intervals and commands. The AAI service allows people with multiple disabilities to control home IoT devices by themselves and to enjoy their spare time on their own by interacting with internet services such as music streaming, Over-the-Top (OTT) media services, and news searching through AI speakers.