Abstract
Virtual reality is playing an essential role in the training system. In this paper, we propose a method to analyze the behaviors of users in the VR training system to evaluate their knowledge about the contents and the performance to finish the trained task. First, the user status in the VR environment such as position, orientation, viewpoint, inputs, and timestamp is recorded. Then, we convert these user status data into spatial-temporal semantic trajectories by projecting the viewpoints into the semantic 3D object in the VR system. To speed up the projection and reduce the noises, the 3D targets are semantically generalized. Finally, an LSTM (long short-term memory)-based algorithm is created to classify the spatial-temporal semantic trajectories of user behaviors. The classification can be used to examine if the user is familiar with the training content. According to our experimental results, the proposed method based on the semantic projection can achieve 85% classification accuracy while the direct LSTM-based classification only has 64%.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bourhim E, Cherkaoui A (2020) Efficacy of virtual reality for studying people's pre-evacuation behavior under fire. Int J Hum-Comput Stud 142:102484
Celikcan U, Askin M, Albayrak D, Capin T (2020) Deep into visual saliency for immersive VR environments rendered in real-time. Comput Graph 88:70–82
Chang Y, Chiao H, Abimannan S, Huang Y, Tsai Y, Lin K (2020) An LSTM-based aggregated model for air pollution forecasting. Atmos Pollut Res 11(8):1451–1463
Ding Y, Zhu Y, Feng J, Zhang P, Cheng Z (2020) Interpretable spatio-temporal attention LSTM model for flood forecasting. Neurocomputing 403:348–359
Harris D, Buckingham G, Wilson M, Brookes J, Mushtaq F, Mon-Williams M, Vine S (2020) The effect of a virtual reality environment on gaze behaviour and motor skill learning. Psychol Sport Exerc 50:101721
Hu C, Duan Y, Liu S, Yan Y, Tao N, Osman A, Ibarra-Castanedo C, Sfarra S, Chen D, Zhang C (2019) LSTM-RNN-based defect classification in honeycomb structures using infrared thermography. Infrared Phys Technol 102:103032
Kang H (2020) Impact of VR on impulsive desire for a destination. J Hosp Tour Manag 42:244–255
Ke S, Xiang F, Zhang Z, Zuo Y (2019) An enhanced interaction framework based on VR, AR and MR in digital twin. Procedia CIRP 83:753–758
Lambe S, Knight I, Kabir T, West J, Patel R, Lister R, Rosebrock L, Rovira A, Garnish B, Freeman J, Clark DM, Waite F, Freeman D (2020) Developing an automated VR cognitive treatment for psychosis: gameChange VR therapy. Journal of Behavioral and Cognitive Therapy 30(1):33–40
Lee J, Eden A, Ewoldsen D, Beyea D, Lee S (2019) Seeing possibilities for action: orienting and exploratory behaviors in VR. Comput Hum Behav 98:158–165
Lee M, Lee S, Jeong M, Oh H (2020) Quality of virtual reality and its impacts on behavioral intention. Int J Hosp Manag 90:102595
Lin J, Cao L, Li H (2020) How the completeness of spatial knowledge influences the evacuation behavior of passengers in metro stations: a VR-based experimental study. Autom Constr 113:103–136
Mao B, Li B (2020) Graph-based 3D building semantic segmentation for sustainability analysis. J Geovis Spat Anal 4:1–12
Miao K, Han T, Yao Y, Lu H, Chen P, Wang B, Zhang J (2020) Application of LSTM for short term fog forecasting based on meteorological elements. Neurocomputing 408:285–291
Pradhan B, Al-Najjar H, Sameen M, Tsang I, Alamri A (2020) Unseen land cover classification from high-resolution orthophotos using integration of zero-shot learning and convolutional neural networks. Remote Sens 12:1676
Rashid K, Louis J (2019) Times-series data augmentation and deep learning for construction equipment activity recognition. Adv Eng Inform 42:100944
Tang L, Ying S, Li L, Biljecki F, Zhu H, Zhu Y, Yang F, Su F (2020) An application-driven LOD modeling paradigm for 3D building models. ISPRS J Photogramm Remote Sens 161:194–207
Vaughan N, Gabrys B (2020) Scoring and assessment in medical VR training simulators with dynamic time series classification. Eng Appl Artif Intell 94:103760
Ying S, Chen N, Li W, Li C, Guo R (2019) Distortion visualization techniques for 3D coherent sets: a case study of 3D building property units. Comput Environ Urban Syst 78:101382
Zheng X, Chen W (2021) An attention-based bi-LSTM method for visual object classification via EEG. Biomedical Signal Processing and Control 63:102174
Zhou F, Hang R, Liu Q, Yuan X (2019) Hyperspectral image classification using spectral-spatial LSTMs. Neurocomputing 328:39–47
Funding
This research has been financed by 2020 Jiangsu Maritime Institute Qianfan plan talent project “Research and Practice of teaching platform of Marine Electronic and Electrical Technology Based on VR technology”, the Jiangsu province university philosophy social science fund project “vocational education reform under the background of transformation and upgrading of Jiangsu maritime vocational education countermeasures study” (2019SJA0644), Jiangsu Maritime Institute Qianfan plan talent project “Research on DP ship FMEA method based on fuzzy reasoning model” (201840), and 2019 Jiangsu High Education Teaching Project “Research and Practice on the cultivation of maritime talents based on ability module” (2019JSJG401).
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible Editor: Biswajeet Pradhan
Rights and permissions
About this article
Cite this article
Li, B., Su, W. Long short-term memory network-based user behavior analysis in virtual reality training system—a case study of the ship communication and navigation equipment training. Arab J Geosci 14, 28 (2021). https://doi.org/10.1007/s12517-020-06312-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12517-020-06312-8