CIRA: An Architecture for Building Configurable Immersive Smart-Rooms

Abstract

Significant technological advancements in computer vision, natural language processing and other fields of computer science and engineering have made way for us to build new human-computer interfaces like a Cognitive Immersive Room (CIR). The CIR described in this work can “see”, “hear”, interpret and respond appropriately to a user or a group of users occupying it. However, bringing the state-of-the-art technologies together to respond to different levels of interpretations of multiple input sources under a unified system that can be easily re-configured to suit different purposes is a challenge. The possible use-cases and benefits of such a smart-room are exciting and limitless. To address this sky-high vision of a smart-room, we describe a software architecture, Cognitive Immersive Room Architecture (CIRA) that allows the room to be programmed for multiple use-cases by providing them spatial and contextual intelligence. In this project, given the relationship of the software with the physical attributes of a room, reconfigurability to various physical environments must be kept in mind. By providing resilience to changes in the lower-level devices, the architecture allows flexibility for researchers to enable different multi-modal interactions. The architecture also supports use-case and context-nesting. Thus, this work describes an architecture that enables a smart-room with multi-modal interaction, interpretation and reasoning. The CIRA supports nested contexts, multiple languages and multiple physical environments. We describe four such contexts (use-cases), two languages and two physical environments prototyped using this architecture and describe a pilot user-study.