Navigation Patterns in Ederly During Multitasking in Virtual Environnment
- 1.5k Downloads
Cognitive assessment and screening can be realized with virtual environments (VE). These VE reproduce ecological situation and give an overview of participants difficulties through scoring systems. The most variables used to qualify participants performance are number of errors and time completion. These variables are link to cognition and navigation skill in VEs. We assessed navigation of adult and elderly in a multitasking VE. Navigation patterns were elaborate with diagram to visually detect differences between the two age groups. Elderly have poorer performance than adults.
KeywordsMild Cognitive Impairment Completion Time Virtual Environment Phone Call Interaction Technique
Cognitive assessment and screening is becoming a major challenge for the researchers to then adapt prevention and support of patients. For instance, to understand Mild Cognitive Impairment’s (MCI) difficulties in their everyday life, they can be assessed during the realization of Instrumental Action of Daily Living (IADL) where errors are analyzed and characterized [1, 2]. Belchior et al.  have listed 9 studies where MCI are assessed with IADL in real world situations. However, IADL can also be tested using virtual environment (VE) [4, 5, 6, 7, 8]. VEs are real time computer simulation in which users can navigate and interact with the use of specific 3D interaction technique [9, 10]. 3D VEs and 2D computerized systems are mostly used in cognitive assessment and screening in dementia or MCI because they offer the possibility to automatically record behaviours in a database and to propose standardized measures . Thus, numerous variables can be monitoring very precisely (e.g. completion time) and collected data can be easily analyzed. Most of the recorded data are linked to participants performance and are related to the usability of the proposed interaction technique. 3D interaction techniques are mainly characterized by 3 terms: navigation, selection and system control . Navigation allows the users to travel in the VE from places to places adjusting his/her view point (i.e. steering)  which is fundamental to explore the VE and to navigate toward the chosen direction [14, 15]. Navigation can be a goal itself or a way to managed a task, but in both cases, it has to be easy to use and to learn to reduce the cognitive resources associated with the motor component (i.e. travel) . So that users can easily be focused on the cognitive component of navigation (i.e. wayfinding) . The imbrication of the travel and wayfinding component foster the creation of a cognitive map of the environment. The cognitive map is a mental representation of spatial information needed to navigate and to orient oneself in space . Thus, navigation data are linked to the action plan and to the interaction technique used for travelling. Using VE permits to progress in the understanding of navigation patterns thanks to the recording of behavioral data. VEs are useful to collect data but the related behavioral analyses are sometime difficult to carry. Indeed, a lot of variables can be selected for analyses and results may be difficult to understand. Even if it is admitted that the users performance is linked to the task completion time  others variables can be extracted and analyzed such as log or time passed in each place of the VE. Having a better understanding of each variables recorded in a VE will allow us to apprehend participant’s performance and adapt his support for instance by modified the interaction technique.
2 Related Work
2.1 Interaction Technique Used in Virtual Environment for Assessing Cognition
Interaction technique used in VE designed for assessing MCI
Virtual supermarket 
Virtual supermarket 
Virtual supermarket 
Touchscreen (push on footprints)
Scrolling left and right on touchscreen
Push on touchscreen
Virtual museum 
Virtual day-out task 
Hand pointing gestures
Hand pointing gestures
Hand grasping gestures (leap motion)
Virtual day-out task 
Treadmill + kinect
Hand grasping gestures (leap motion)
Virtual park 
Serious game 
Push on touch screen
To understand the impact of interaction technique on the participants performance it is necessary to give an exhaustive description of the system setup and the interaction technique used. The view point orientation can miss in the system description whereas this characteristic is primordial. Indeed, when a participant is navigating in a VE he must be able to move easily, looking around him . Moreover navigation must be very easy allowing the participant to completely focus on the tasks to be achieved within the VE .
The challenge of human-machine interaction studies is to developpe and evaluate interaction techniques with a high usability and to understand how cognition is involved in the use of these interaction technique. Indeed, studies have shown that elderly and MCI can be uncomfortable with the use of mice  and can feel anxious about using it . However, if some elderly has difficulties with technology acceptance, increasingly are using computer for instance to keep social link thanks to e-mail . The use of computer and associated interaction technique such as mice and pads, is becoming very common in work as in private context. Moreover, mice could be used for screening MCI. Seelye et al.  found, with mouse data monitoring during web searching, that MCI make more resting time between mouse movements. Taken together these studies indicates that elderly can use various interaction technique and that navigational data could help to qualify users performance.
2.2 Cognitive Assessment in Virtual Environments
The use of VEs to assess cognitive functions specify to pick up variables used to score the performance of the participant. Indeed, these new tests recquire to readjust the scoring system from traditional tests especially in IADL tests. VEs permit to record a lot of variables but investigations are still needed to detect which variables are sensitive to correctly qualify the performance. Moreover, the recorded data are related both cognition and the use of interaction techniques. Indeed, the main variables used to qualify performance are completion time and number of error which are modulated by navigation efficiency in the VE. Navigation efficiency is mainly assess by completion time and so, linked to the user’s performance  where a long completion time is associated to a poor performance. These results are corroborated with some studies where MCI took significantly more time to complete the task than experimental participants [5, 6, 7]. For example, the navigation patterns can be shown with the Visualization of Users Flow (VU-Flow), a tool  which allow to analyze navigation patterns in 3D VE. VU-Flow records users position and orientation in logs so users path can be visualized as well as most visited routes. The path visualization leads to descriptive and quantitative data of navigational efficiency with fluid or saccadic paths and distance travelled . Even if tools exist to visualize navigation data, it is still difficult to understand why patients took more time to realize the task. Is it because they are inefficient in the task realization (i.e. cognitive difficulties) or because the use of the interaction technique?
Variables used to assess participants performance in the 8 VE (cf. Table 1) were listed. Two methods can be used to qualify performance: created an efficiency ratio which is the compilation of several variables [8, 25] or analyzed variables one by one where most variables are time completion and number of errors. Others collected data exist as number and rest periods [4, 5] or distance travelled [4, 5, 6]. So, variables used in VE to qualify participants performance are related to quantitative navigation data. Mainly because monitoring data are linked to the tracking of participants position. Even if these data are difficult to appreciate because they are related to the use of the interaction technique, they can be reliable to qualify participants performance. For instance, Zygouris et al.  proposed an algorithm to detect MCI participant from elderly according time completion and age of the participants where a longer completion time is a sensitive variable to detect participants with MCI.
The present studys goal is to apprehend participants performance focusing on navigational data in a 3D EV. Results could help to have a better understanding of elderly performance and later compared them to MCI performance.
The study was conducted with 13 young adults (4 men and 9 women) and 13 elderly (3 men and 10 women). Young participants (mean age: 25.16, SD: 6.14) were recruited in general population and elderly (mean age: 67.09, SD: 2.91) at the Alzheimer association. All participants are French speaker and have the same year of scolarity (mean 16.12 years for young adults and mean 15.25 years for elderly). Older with a MoCA less than 26 were excluded on this study. The participants realized IADL tasks during multitasking in the Virtual Multitasking Task-2 (VMT-2). The VMT-2 is a virtual apartment where prospective memory and executive functions are assessed during IADL. All trials begin with a familiarization stage where participants could visit the VMT-2, displayed on a full HD 27” monitor, without tasks to manage using mouse and keyboard and head-mounted display (HMD) (i.e. Oculus Rift). The goal of this preliminary step is to allow participants creating a cognitive map of the virtual apartment and to learn how to navigate select 3D objects (e.g. open/close a door). Then, all participants realized VMT-2 while navigation data and action were recorded in a csv database. Participants position was monitored in the x, z and y axis and transcribed on the apartment map. Collisions are marked using a red triangle on the map each time the user touch a 3D object such as a wall or a furniture. Total completion time and the time passed in each room of the virtual apartment were also recorded. Each participants had to perform the familiarization step in both a non-immersive and an immersive configuration. In the former case, the participants hat to navigate using mouse (travel) and keyboard (steering and selection), in the later case they had to navigate using the keyboard (travel), the HMD (steering) and the mouse (selection).
When the participant begins the VMT-2, he knows that he must answer to the phone if this one is ringing and to store the grocery. One minute after the beginning the phone rings and a voice asks the user to go to the main bedroom to feed the fishes and to dry on a shirt. Two minutes after the beginning, the voice on the phone asks to fax a document. Finally, after 15 min it asks to check if the tickets are properly placed on a shelf. The experiment ends when the participant think he has finished the tasks.
All analyses were conducted using R software, at a significance level of 0.05. T modified was used to qualify errancies and U Mann-Whitney to compare elderly from young adult group.
4.1 Descriptive Analysis
4.2 Quantitative Analysis
Time Passed in Each Zone. Time passed in each zone was calculated to identified errancies. Errancy can be defined as a long time passed in one room including goings and comings without apparent goal. To qualify errancies, T modified test was conduct so that participant resting significantly more time than others in a room was doing errancy. There is no significantly difference of errancies between the 2 groups (U = 83; p = 0.47) or between immersion conditions (i.e. HMD and non-immersive) (U = 41; p = 0.96).
Completion Time. Elderly group took significantly more time to achieve the VMT-2 (U = 126; p = 0.009). We did not observe any effect of immersion (U = 82; p = 0.40).
This study investigated navigation patterns of elderly and young adults while performing multitasking in a virtual apartment, by analyzing data such as completion time. In this context, navigation through each room has been analyzed using a diagram. When participants begin task they know they have to go in the kitchen to store the groceries but phone calls interrupted them during their first action. The call ask to realize others tasks in several rooms of the virtual apartment. The addition of successive tasks will distract the subject but also ask him to locate himself in space to be as efficient as possible in carrying out the tasks. To do it and complete the VMT-2, young adults took significantly less time to complete the tasks than elderly. It is now to understand why and where elderly need more time and what difficulties did they encounter.
Elderly took more time to realize the tasks in the VMT-2 which is congruous with previous studies where older participants had higher time completion to manage navigation tasks . In their study, Sayers et al.  proposed some navigation aids to increase participant’s spatial awarness. However, differences between young adults and elderly were maintain. Elderly may have issues with the use of the interaction technique itself and not only with space awareness. Indeed, participants begin VMT-2 with a familiarization stage where they can visit the VE once with HMD and once with mouse and keyboard. Before being assessed in VE, participants generally begin with a familiarization step where they can learn how to interact with the VE. Being familiar with the VE is also often required for a successful navigation . The recorded data during these early stages could be a reference for the participant’s skill using the interaction technique to classify them according the user’s abilities such as expert or novice. Indeed, during the familiarization step, paths with mouse are less saccadic than paths with HMD for both adults and elderly and participants made more collisions with HDM during the experimentation stage. When using the HMD, participants have non-continuous movements. They stop travelling, then look in the wanted direction (i.e. steering) and move on again. This lake of spontaneity is shown by right angles marked on the virtual apartments map (Fig. 1). Moreover, most of participants decided to travel with mouse and keyboard during the experimentation because they felt uncomfortable using the HMD. Interaction technique can modulated the participant’s performance especially in the case where steering and travel are two different processus. Indeed, if travel is the same in both condition, steering is doing by mouse or by movements of the head, with changing orientation point of view. Steering with head movements is not intuitive for participant without experience in immersive VE and could lead to higher cognitive load for HMD users.
Aging is associate with cerebral change that may cause mild decline in attention, executive functions and working memory . These declines may make elderly slower at performing tasks. As expected, elderly are less efficient than adults in the realization of tasks in VE. Indeed, they took more time to initiate tasks in the VMT-2 and half of elderly begin activities only after the second phone call. During the first and the second phone call, they stay in the same zone and seem to use the time to think. As they need more time to initiate activities in the end of the task the had several tasks to finish during the same time, so they are doing multitasking to manage all tasks. The multitasking context is present in IADL and is linked to independent living at home . It requires several cognitive functions as executive functions, prospective and retrospective memory [37, 38, 39] and can lead to a higher cognitive load and difficulties to remember all the things to do. Indeed, most of elderly forget to realize the last task which is to check the tickets. Moreover, the multitasking context with unexpected events could lead to errancies for adults and elderly. During this time, they may think about the way to managed the next tasks. Indeed, when checking logs, we recorded any activities of the participant during errancy. However, we also observe another type of long time passed in a zone especially in the corridor or in the kitchen. According logs, in these room, participant had issue with item selection and orientation in space in these restricted places.
Taking together these results indicate that the elaboration of a EV for assessing cognitive functions is not an easy thing. Indeed, results showed that elerdly realized tasks in an inefficient way because phone calls were unexpected and disturbe participants but also because they need more time to be ready to act in the VE. In the VMT-2 the first phone call seems to occurs too early because the participants do not have the time to go in the kitchen to store the groceries. Moreover, adult participants finish the task quickly and the third phone call did not ring. Start actions in VE as function of time could lead to difficulties for participants to managed all the task. Actions could be launch not by time but by others actions. For instance, the first phone call ring after participant stored some grocery item instead of 1 min after the beginning of the VMT-2. A longer completion time is associated with cognition but also with the interaction technique. Analyze several variables instead of having a performances rate helps to make a distinction between cognition and abilities with the interaction technique. Understand how which part impacts more completion time could be make with analysis of collisions, time passed in each zone and logs. Collisions could be a quantitative indicator of usability of interaction technique and logs visualized with a diagramm could be a better indicator of participant’s cognition. Analyze logs to construct diagram aims to be a useful method which allow to see where and what kind of difficulties the participant met.
As limitation, the study has not enough participant using the HMD. Complementary analyses should be conduct with more participants wearing the HMD. To avoid cybersickness with elderly the comparison could be done only with adults.
Thank to all students who work hard to make this research possible.
- 4.Josman, N., Klinger, E., Kizony, R.: Performance within the virtual action planning supermarket (VAP-S): an executive function profile of three different populations suffering from deficits in the central nervous system. In: Proceedings of the 7th ICDVRAT (2008)Google Scholar
- 6.Zygouris, S., Giakoumis, D., Votis, K., Doumpoulakis, S., Ntovas, K., Segkouli, S., Karagiannidis, C., Tzovaras, D., Isolaki, M.: Can a virtual reality cognitive training application fulfill a dual role? using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. J. Alzheimers Dis. 44(4), 1333–1347 (2015)Google Scholar
- 7.Tarnanas, I., Laskaris, N., Tsolaki, M., Muri, R., Nef, T., Mosimann, U.P.: On the comparison of a novel serious game and electroencephalography biomarkers for early dementia screening. In: GeNeDis (2015)Google Scholar
- 9.Stanney, K.M., Kennedy, R.S., Kingdon, K.: Virtual environment usage protocols. Handbook of Virtual Environments: Design, Implementation, and Applications. Lawrence Erlbaum Associates Inc., New Jersey (2002)Google Scholar
- 11.Morris, R., Kotitsa, M., Bramham, J., Brooks, B., Rose, F.: Virtual reality investigation of strategy formation, rule breaking and prospective memory in patients with focal prefrontal neurosurgical lesions. In: Proceedings of the 4th International Conference on Disability, Virtual Reality & Associated Technologies (2002)Google Scholar
- 12.Jankowski, J., Hachet, M.: A survey of interaction techniques for interactive 3D environments. In: Eurographics 2013-STAR (2013)Google Scholar
- 14.Liu, A., Fung, J., Lamontagne, A., Hoge, R., Doyon, J.: Brain regions involved in locomotor steering in a virtual environment. In: ICVR (2013)Google Scholar
- 16.Bowman, D.A.: Interaction Techniques for Immersive Virtual Environments: Design, Evaluation, and Application. Invited Talk (1997)Google Scholar
- 18.Tolman, E.C.: Cognitive Maps in Rats and Men. American Psychological Association (1948)Google Scholar
- 19.Patel, K.K., Vij, S.K.: Spatial navigation in virtual world. In: Advanced Knowledge Based Systems: Model, Applications and Research, TMRF e-Book, vol. 1, pp. 101–125 (2010)Google Scholar
- 20.Jung, J., Park, H., Hwang, D., Son, M., Beck, D., Park, J., Park, W.: A review on interaction techniques in virtual environments. In: 2014 International Conference on Industrial Engineering and Operations Management, pp. 1582–1590 (2014)Google Scholar
- 21.Poupyrev, I., Ichikawa, T., Weghorst, S., Billinghurst, M.: Egocentric object manipulation in virtual environments: empirical evaluation of interaction techniques. Comput. Graph. Forum (1998)Google Scholar
- 24.DSouza, L., Pathirana, I., McMeel, D., Amor, R.: Kinect to architecture. In: Twenty-Sixth International Conference Image and Vision Computing, pp. 155–160, New Zealand (2011)Google Scholar
- 28.Fischer, L., Oliveira, G., Osmari, D., Nedel, L.: Finding hidden objects in large 3D environments: the supermarket problem. In: Virtual Reality (SVR) XIII Symposium, pp. 79–88. IEEE (2011)Google Scholar
- 32.Zickuhr, K., Madden, M.: Older adults and Internet use: For the first time, half of adults ages 65 and older are online. In: Pew Internet & American Life Project, Washington (2012)Google Scholar
- 33.Seelye, A., Hagler, S., Mattek, N., Howieson, D.B., Wild, K., Dodge, H.H., Kaye, J.A.: Computer mouse movement patterns: a potential marker of mild cognitive impairment. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 1, 472–480 (2015)Google Scholar
- 35.Park, D.C.: The basic mechanisms accounting for age-related decline in cognitive function. Cognitive Aging: A Primer 11, 3–19 (2000)Google Scholar
- 37.Burgess, P.W.: Real-world multitasking from a cognitive neuroscience perspective. In: Control of Cognitive Processes: Attention and Performance, pp. 465–472 (2000)Google Scholar