Emotional Effect of Multimodal Sense Interaction in a Virtual Reality Space Using Wearable Technology

Abstract

The virtual reality (VR) market has been expanding fast with the extraordinary progress of relevant hardware and software. Particularly, with the commercialization of standalone head-mounted display (HMD) devices in the recent years, there has been an increasing amount of interest in interfaces providing more expanded and sensual information to users. To intensify user immersion in a VR space, it is necessary to provide physical sense information similar to a real world. However, the sense information provided to users is limited to mostly simple vibration or force feedback, and the emotional responses of users have not yet been investigated with respect this information. Therefore, this study presents the approach of providing multimodal sense feedback to a user in a VR space through the “Emoract” interface that can be worn on both hands and also investigates the resulting emotional responses by measuring the user’s biological response signals. All the experiments were conducted using an interactive VR animation “Lonely Noah”. In the VR animation, because the user’s emotional response to the user’s sensory response changes depending on the story and situation, there is an effective interaction with vibration sensations such as the existing VR interface, and there is an effective iteration with context-based multi-modal sensory feedback rather than vibration.