Abstract
Virtual reality (VR) is a computer-simulated three-dimensional environment whose goal is to achieve a strong sense of being there, called immersion. This requires a variety of technologies: the most essential being input devices. At present, the non-contact Kinect somatosensory interaction approach has been widely studied. Somatosensory interactive games take body movements as input instructions, instead of the simple push-button operation mode. Hence, the operational behavior is more intuitive and the reaction of players is highly accurate and brings a better game experience to players. In terms of man–machine interaction in somatosensory interactive games, gesture interaction brings the primary operation medium into the virtual environment. This paper analyzes the problems of somatosensory games interaction in immersive virtual reality, focusing on gesture-based immersive virtual reality technology. The main work includes: First, aiming at the problem that the gesture recognition method cannot adapt to the real-time continuous gesture flow in practical application scenarios, a naive Bayes gesture recognition algorithm based on joint vector is proposed. The algorithm uses a decision support system for feature representation of bone and motion along with the joint vector. It introduces the adaptive naive Bayes algorithm, which can complete the training with fewer samples. Experimental results show that the algorithm has a high recognition rate and recognition speed, can effectively adapt to continuous data flow, has good compatibility with virtual reality applications, and significantly increases the user's sense of immersion in virtual scenes. Secondly, in view of the lack of complete visual feedback solutions for gesture interaction, a 3D user interface and interaction method combining bone animation and collision detection technology is proposed. The process interacts with the virtual environment through gesture feedback from 3DUI elements. Experimental results show that the design can effectively enhance the immersive experience through the visual feedback of gestures.
Similar content being viewed by others
References
Accot J, Zhai S (1997) Beyond Fitts’ law: models for trajectorybased HCI tasks. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems, ACM Press, New York, 295–302
Cai L, Zhang J, Liu B (2015) Somatosensory interaction with virtual environments based on gesture recognition. J Huazhong Univ Sci Technol (Natl Sci Ed) 43:136–139
Chapoulie E (2014) Gestures and direct manipulation for immersive virtual reality. University of Nice Sophia Antipolis, France
Detlefsen J (2014) The cosmic perspective-teaching middle-school children astronomy using ego-centric virtual reality. Aalborg University, Denmark
Goude D, Bjoerk S, Rydmark M (2007) Game design in virtual reality systems for stroke rehabilitation. Stud Health Technol Inform 125:146–148
Ivan ES (1968) A head-mounted three-dimensional display. In: Proceedings of the 1968 fall joint computer conference. ACM Press, Seattle
Kikuta F, Fukaya K, Takizawa T et al (2002) The development of “role-playing game for healthy life through computed virtual reality in dialogical pattern” for experiential self health. IEICE Tech Rep Educ Technol 101:39–43
Kim JY, Nam SH (2014) A study of immersive game contents system design and modeling for virtual reality technology. Int J Control Autom 7(10):411–418
Law EF, Dahlquist LM, Sil S et al (2011) Videogame distraction using virtual reality technology for children experiencing cold pressor pain: the role of cognitive processing. J Pediatr Psychol 36(1):84–94
Li H, Ding J, Wu Y et al (2015) Static three-dimensional gesture recognition based on kinect skeleton data. Comput Appl Softw 32(9):161–165
Liang XB, Wang C, Zhen W (2016) A somatosensory interaction system based on kinect. MATEC Web Conf 44(5):02024
Liu X, Tian L, Zhang C, et al (2019) Interactive motion system of intelligent somatosensory virtual reality. Technology Innovation and Application
Lloréns R, Navarro MD, Alcañiz M, Noé E (2012) Therapeutic effectiveness of a virtual reality game in self-awareness after acquired brain injury. Stud Health Technol Inform 181:297–301
Ma ShuG (2013) The research of next-gen game engine virtual reality in actual project. Appl Mech Mater 443:39–43
Mangera R (2013) Static gesture recognition using features extracted from skeletal data. In: Proceedings of the 2013 annual symposium of the pattern recognition. PRASA, Aukland, 59–63
Nancel M, Wagner J, Pietriga E, et al (2011) Mid-air pan-and-zoom on wall-sized displays. In: Proceedings of the SIGCH conference on human factors in computing systems, ACM Press, New York, pp 177–186
Paul MF (1954) The information capacity of the human motor system in controlling the amplitude of movement[J]. J Exp Psychol 47(6):381
Schultheis MT, Rizzo AA (2001) The application of virtual reality technology in rehabilitation. Rehabil Psychol 46(3):296–311
Shin JH, Ryu H, Jang S (2014) A task-specific interactive game-based virtual reality rehabilitation system for patients with stroke: a usability test and two clinical experiments. J Neuroeng Rehabil 11(1):1–10
Slater M (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc B 364(1535):3549–3557
Slater M, Usoh M, Steed A (1995) Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans Comput Hum Interact 2(3):201–219
Suresh R, Riead LH, Straub J, et al. Using virtual reality and game technology to assist command and control. In: Society of photo-optical instrumentation engineers. Society of photo-optical instrumentation engineers (SPIE) conference series, 2017:1020508
Youngblut C (1998) Educational use of virtual reality technology. Ida Document D, 2128
Zakirova AA, Ganiev BA, Mullin RI (2015) Using virtual reality technology and hand tracking technology to create software for training surgical skills in 3D game. AIP Publishing LLC, New Operational Technologies, New York
Zeroth JA, Dahlquist LM, Foxen-Craft EC (2018) The effects of auditory background noise and virtual reality technology on video game distraction analgesia. Scand J Pain 19:207–217
Zhu C, Lu Y, Jiang L, et al (2021) Liquid crystal soft actuators and robots toward mixed reality. Adv Funct Mater, 2009835.
Funding
The paper did not receive any financial support.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author declared that he has no conflicts of interest to this work.
Ethical approval
My paper does not deal with any ethical problems.
Informed consent
I declare that I have informed Consent.
Additional information
Communicated by Tiancheng Yang.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gao, P. Key technologies of human–computer interaction for immersive somatosensory interactive games using VR technology. Soft Comput 26, 10947–10956 (2022). https://doi.org/10.1007/s00500-022-07240-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-022-07240-3