Advertisement

Using Virtual Reality to Assess the Elderly: The Impact of Human-Computer Interfaces on Cognition

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10274)

Abstract

Prospective memory (PM) is defined be the capacity to remember to realize an intended action in the future. This is a very important cognitive function that permit to maximize autonomy in everyday life. Unfortunately, few assessment tool, valid, reliable and ecological is accessible for clinicians. To obtain a verisimilar and ecologically prospective memory assessment tool, virtual reality seems to be a promising way. A specific and sensible tool could help the clinician to detect subtle changes in the cognition of the elderly and, ideally detect pathological aging soon before the beginning of decline. Because older adults are not really at ease with technology, these (dis)abilities could be confounded with cognitive inefficacy and lead to false positives diagnostics. To avoid this, the psychometrician must consider the impact of human-computer interfaces (HMI) on cognition. This paper present three experiments that show the impact of HMI on stress, capacity to achieve a task and on cognitive load. The first pilot study shown that a “heavy to use” HMI generated stress and difficulty to achieve the task with healthy adults. The second pilot study revealed that VMT-2 is judged moderately challenging cognitively and it seems to be more for older participants. The third pilot study shown that a complex virtual environment (in terms of navigation and interaction) is more cognitively challenging than a simple virtual environment for older peoples compared to young participants. These results indicated the importance of considering HMI as a potential variable that could create bias in the cognitive measurement.

Keywords

Virtual reality Human-computer interface elderly Assessment Prospective memory Cognition 

1 Introduction

Prospective memory (PM) is a set of cognitive functions responsible for our ability to remember to carry out an intended action at a specific time in the future [1]. It is now recognized that proper functioning of the prospective memory is essential in everyday life because it gives people a maximum of independence in carrying out normal activities. By providing a certain degree of personal safety in everyday life, prospective memory also has a direct effect on people’s quality of life and on their social participation. Some activities governed by prospective memory include remembering to take medication, arriving at an appointment on time and turning off the stove after using it [2]. PM is a major set of cognitive function to allow an independent life. Indeed, three types of remembering based on the nature of prospective cues have been described: Event-based, Activity-based and Time-based [3]. Event-based PM tasks are facilitated when a cue is present in the environment. The best example is the person who wishes to buy orange juice on the way home: here prospective remembering “pops up” with the appearance of a grocery store. Activity-based prospective memory consists of achieving a specific action (for example, taking a medication) in association with the realization of another action (for example, when the person is going to the work). Finally, Time-based prospective memory tasks consist of remembering an intention at a specific time (e.g. 10 o’clock) or after a specific delay (e.g. in a 20-minute delay). A good example of a Time-based prospective memory event is meeting a colleague at a specific time and day or removing a cake from the oven after 30 min.

Assessing PM is primary to understand patient’s difficulties in everyday life. For instance, aging population reports frequently PM problems as evaluate by some laboratory tasks [4] but PM problems seem to be difficult to observe in the everyday life. This particularly the case when patients are performing during natural tasks (a) realized in a familiar context, (b) interfolded in everyday life and (c) executed on several days. This contrast into the results could be explained by anxiety generate in a laboratory setting or by the fact that the elderly could use compensatory strategies in the everyday life such as a calendar, Post-it, etc. Here, virtual reality (VR) could be a good means to assess prospective memory in a verisimilar ecological manner with standardized procedures. VR is a computerize simulation of the real world. It allows to propose virtual environment (VE) designed fully to assess cognitive function in everyday life without unwanted distractors or possibilities for the patient to use his familiar compensatory strategies.

Authors have proposed an interesting model of PM (The Multiprocess Model) [5]. This model argues it has 2 main modes of remembering: automatic and strategic. More specifically, when people achieve a prospective task under automatic processes, the contact with the cue in the environment is sufficient to remember the attended action. In fact, this remembering could depend more on environmental cues, thus soliciting more memory and attention than executive functions. Strategic remembering implies bigger cognitive resources because it supposes a strategic and voluntary monitoring between ongoing and prospective tasks [5]. This process might be supported by the Supervisory Attention System (SAS), an attention-executive process [6]. The SAS facilitates encoding an association between an external event and the intended action. After, the SAS monitors the environment and searches the targeted cue to indicate the right time to perform the action [1, 7]. When the target is identified, the SAS interrupts the ongoing task, turns the attention toward the prospective task and lets the person realize the intended action. In summary, the role of the SAS in the Multiprocess model is to support the realization of the action plan by the activation of the intended scripts and by diminishing the non-pertinent scripts; this process could be implied also when the individual performs in attention mode. The SAS implies that attention resources are limited in “energy”. Actually, when attention is focused on one element of the entire task, it is difficult to pay attention to the other part of the task. In VR, we could think that when participant is concentrated on the navigation with an external device (e.g. joysticks, mice) into the virtual environment (VE), his/her cognitive readiness for the task are reduced. That could create a bias in the measurement obtained into the VE.

Because PM is a cognitive function weakened in pathological aging such as mild cognitive impairment [8], it is very important to consider the best way to assess it efficacy. Several authors said the PM must be systematically assessed by neuropsychologist when a diagnostic of dementia is suspected [9]. Unfortunately, very few assessment and clinically useful tool for PM exist because some of them lack reliability. VR could be a better alternative to assess PM but with these tools it is essential to be able to understand which part of the cognitive load is solicited by the Human-Machine interface (HCI).

1.1 Assessing Prospective Memory

The ecological validity of measures of cognitive function is contingent on the functional and predictive relationship between an individual’s performance on a set of neuropsychological tests and their real-life behaviour at home, at work, at school or in the community [11]. In other words, a test is ecologically valid if it is able to predict an individual’s level of function or detect potential problems in his/her daily life. The major challenge concerning a new paradigm in neuropsychological assessment is to develop tools with a good face-and-content validity (i.e. tasks should more closely resemble those of daily life) and with predictive validity (i.e. test must be correlated with the individual’s real-life functioning). From a psychometric perspective, ecological validity is represented by the link between the results observed in a test and the capacities of the subject as it is seen in everyday life [12]. Two dimensions of ecological tests have been reported [13]; that is useful in the understanding of a cognitive assessment from an ecological perspective. To sum up, the test can be constructed or reviewed from a “veridicality” or “verisimilitude” perspective.

Veridicality concerns all existing traditional tests and the way which they are linked empirically with everyday life [12]. Originally, the test had not been created to simulate daily-living activity. On the other hand, it can be predictive on how cognitive functions will help or not in the realization of an activity. The Trail Making Test (TMT), attention and executive functions test, is a good example of this kind of assessment. The theoretical construct of the TMT is not ecological, but it can be useful in terms of prediction. Here, it is important to remember that a test having good diagnosis capacities is not necessary ecologically valid. Moreover, [13] have found some contradictions concerning correlations obtained between some traditional tests (e.g. WCST, TMT, Controlled Oral Word Association Test) and everyday performance in naturalistic tasks that imply executive functions. In brief, veridicality is a field of ecological assessment which claims that it is possible to use a traditional neuropsychological test in order to predict everyday performance; in the end, further research in this field as far as current literature is concerned must be conducted.

Verisimilitude represents the capacities of a test to have the same cognitive demands as those found in everyday life from a theoretical perspective [12]. This approach required reinventing neuropsychological assessment in order to create a new assessment protocol, nearer to the reality of the person [24]. In this way, the expectation was to have a test with better content and criterion validity compared to a traditional one. Efforts such as those mentioned above, which set out to create every day-like tests, make better the distinction between verisimilitude and veridicality of the neuropsychological assessment.

A very good verisimilar neuropsychological test is the Multiple Errands Test (MET), which was initially developed by [14]. In this task, participants are asked to carry out tasks in a real environment. For example, the subject has to purchase some items and spend less money as possible, to avoid buying non-asked items, to respect some arbitrary rules (e.g. “Don’t talk to the evaluator during the task;” “Don’t exit a definite perimeter,” etc.). Although the MET has been shown to detect subtle problems of day-to-day living, it lacks reliability from a psychometric point of view [15]. Because the test is naturalistic, results obtained from the test in real-life settings are difficult to reproduce from individual to individual and from situation to situation [16]. But some authors have argued these psychometric considerations are contoured by a good inter-rater reliability [16, 17]. The main advantages of the MET are its sensitivity to specific neuropsychological deficits and its ecological validity [15].

From a neuropsychological perspective, the main challenge to developing new assessment tools is the standardization of assessment procedures that contain naturalistic and plausible tasks regarding day-to-day life. The development of ecological neuropsychological tests leads to the review of test structure [18]. However, if the task implies real activity and interactivity with daily-living activities, then it could lack reliability in the way of repetition of measure. Therefore, some authors have tried to create new assessments performed in the office of the neuropsychologist with an ecological construct.

Several situations in the everyday life demand good prospective memory capacities [19]. It seems to be difficult to have ecological and “verisimilar” measures of PM. To reproduce day-to-day living in a more realistic way and to assure standardized, reliable, and valid measures, several researchers have explored the potential of VR technology. In VR, the user navigates in a computer-simulated environment and interacts with objects in real time, ‘like in real life’ [20]. This technology offers researchers the best of both worlds [18]: it allows them to observe real-life situations in a laboratory setting [21]. There are several advantages that make it attractive to those in the field of neuropsychological assessment. Firstly, it allows access to the construct of prospective memory in a systematic, rigorous, and standardized manner [21, 22, 23] while at the same time providing a degree of ecological realism [24]. Also, because the effects of extraneous variables commonly encountered in real life can be controlled, VR researchers can assess the effects of standardized ‘unexpected situations’ in these environments [21].

VR is therefore considered as an ecologically valid tool to assess cognitive functions involved in everyday life. Studies in our group have shown the capacities of VR to detect cognitive dysfunction after traumatic brain injury or dementia [25, 26]. We have also demonstrated significant correlations between performance on virtual tasks and neuropsychological measures [27, 28]. However, our works have also demonstrated that HCI can generate a cognitive overload which affects cognitive functioning and creates a measurement bias. In accordance with our finding, some authors have demonstrated that VE can generate cognitive overload, mainly for people who are not familiar with video games, which is the case for the elderly [29]. Others have also found a significant cognitive overload in students performing tasks in a VE experienced [30]. These students reported having difficulty to direct their attention, to keep in mind several sources of information when exploring the virtual world. Therefore, there would be several active cognitive processes in the foreground and in the background when it comes to using a VE to evaluate the cognitive skills of an individual [31], particularly in the mobilization of cognitive resources by HCI.

1.2 Objectives

To investigate the effect of HCI on cognition during a neuropsychological assessment with VR, we have conducted three complementary pilot studies. All these studies implicated participants without brain lesions or psychiatric history. The purpose of these studies was to assess the normal functioning of an adult doing tasks in a VE; a magnifying glass will be focused on older participants. All these studies were conducted with the Virtual Multitasking Test (VMT) [32].

2 Method

2.1 Pilot Study 1

The aim of the first pilot study was to assess the effect of a technology known as “heavy use” (i.e. disturbs the navigation and interactions with the VE) on tasks realization during immersion in the VMT.

Participants. Five healthy adult participants (2 women & 3 men) were recruited into the general population. The mean age was 34, 5 (SD: 12,07) and mean years of education was 15 (SD: 1,55).

Virtual environment.

The Virtual Multitasking Test (VMT) aims, at the beginning of its development, to assess PM and executive functions using a multitasking paradigm [33]. Different scenarios are implanted into a 6 ½ rooms virtual apartment, each room including at least one task except the bathroom. At the beginning of the test, participants are told that they are visiting their best friend. During the day, he is at work and they must live in his apartment. In the evening, they will go to a show with their best friend. However, during the day, they must perform several tasks alone based on daily life. For instance, they must store the groceries on the counter as quickly as possible (even if they are told there is no time limit to complete the activities), answer the phone, and perform other tasks such as faxing a document, search for show tickets, dry a shirt, feed a fish. PM tasks require, among other things, to close a door just when exiting the master bedroom to prevent a dog from climbing on the bed. Unforeseen events occur during the execution of the tasks. For instance, the occurrence of a storm which overthrow objects in the guest room and let water seep into the dining room. For example, storms that reverse objects in the guest room and that let water seep into the dining room. Every time a person is exposed to the VMT, they start a training phase of the environment. Afterwards, the experimental phase began and the person had to carry out the tasks proposed by the scenarios planned by researchers.

Materiel.

A Head Mounted Display (eMagin z800) was used in this experimental setting that got the head movements and allowed immersion. Participants wore a 6DOF sensor on the dominant hand (Ascention’s Flock of Bird) to manipulate objects in the VE. They had in their other hand a mouse that guaranteed their movements and they had to manipulate the right, left and centre buttons to perform certain actions in the VE. The experiment was realized standing to find consistency and keep the realism of the proposed scenario. Blood pressure and heart rate measurements were also taken every 5 min through an Ambulatory Blood Pressure Monitor (ABPM) (Whelsh Allyn) to evaluate the biological variations, i.e. probable stress indicators or measurement of the workload during the task completion.

Results.

In the first experimental setting, participants felt a little stressed or anxious in relation to the upcoming experimentation (2.33 ± 2.7 on a scale of 10), and blood pressure was in the limit of normal at the time of starting the experimental protocol. After immersion, they reported that the interaction with the VMT was difficult and frustrating (8.40 ± 2.07 on a scale of 10). The analysis of individual performance demonstrated that no participant came to efficiently complete the task and that there was a great variation in the immersion time (total time of immersion: 19.45 ± 17.55 min), indicating a rise of workload during the immersion. Moreover, many mistakes were made and tasks were performed in a none-efficient way. Concerning the reported biological measures, three out of five (60%) participants showed a higher level of blood pressure during the immersion thus achieving prehypertension thresholds (120/80 mmHg) compared to initial measure. Heart rate was also affected, rising above 80 beats per minute for two participants (40%; while the normal threshold at rest was 70 beats per minute). However, despite these observations and changes of the participant’s internal state, the average sense of presence has remained satisfactory (7.67 ± 0.816/10).

2.2 Pilot Study 2

The aim of this study was to explore the effects of mental workload generated by the VMT second version.

Participants.

Thirteen healthy adults were recruited into the general population to participate in this study. Six were aged between 18 and 45 years old (mean: 30; SD: 8,83) and 7 were aged over 65 years old (mean: 67,25; SD: 2,87). The experiment took place on two non-consecutive half days: the first one to take neuropsychological assessment to evaluate the executive functions, which involves “traditional” tests such as Delis-Kaplan Executive Function – System (D-KEFS) and the California Verbal Learning Test (CVLT) and “ecological” tests such as the Behavioural Assessment of the Dysexecutive Syndrome (BADS) and the Rivermead Behavioural Memory Test (RBMT) and the second to realize several tasks in the VMT.

Virtual environment.

Based on the results obtained in the first pilot study, to conduct this second study, VMT was migrated from VIRTOOL towards UNITY 3D to simplify HCI and to make the task in the VE more fluid. Several modifications were brought to the original scenario to complexity and made it more valid on a theoretical point of view the assessment.

Procedure.

The experimental procedure was divided in two phases (learning and test) of the new VMT-2, which was used in a non-immersive mode. The necessary equipment for the study was a computer with a multimedia projector, a keyboard for movement and mouse for gripping objects. At the end of the experiment, eight participants completed the entirety of the experimental protocol. Attrition was mainly explained by the length of the experiment (duration when including the two phases: 6 h under two days). The latter fact could possibly explain the dropout from the study.

Results.

At the end of the experiment, the participants didn’t experience cybersickness (sum = 4.43 ± 3.78/64). The feeling of presence was, for its part, in the average (mean = 2.47 ± 0.45/4). On the basis of the NASA-TLX questionnaire, VMT-2 appeared to induce a relatively modest cognitive load (i.e. 46% of estimated cognitive load). However, considering the standard deviation obtained, it seemed that some people are more likely to judge the environment as rather demanding. Indeed, it seemed that it was mainly the case for those aged over 65 years old.

2.3 Pilot Study 3

This third and final pilot study want to explore, while simplifying the HCI, if there was an age-related difference between young and elderly adults: (a) when the requested tasks in a VE were more complex and (b) on the user’s experience with the interactive system design (UX) based on the VE used. In sum, the experimental design consisted of age-related comparison of the cognitive load caused by two different VEs.

Participants.

Nine young adult (18–45 years; mean = 30.44 years ± 4.98 years) and 8 elderly participants (55 years and up; 68.38 ± 9.13 years) were recruited from the general population. Hence, to be eligible for this study, participants must not have a neurological or psychiatric history; for the elderly, the Montreal Cognitive Assessment (MoCA) should be fewer than 26. Neither group differed significantly on the education plan [F(1,15) = 3.07; p = 0.1]. When asked to give a percentage (/100%) on the level of comfort use with computers, young adults (80 ± 17.79) expressed more comfort compared to elderly adults (55.63% ± 36.79); [F(1,15) = 3.36; p = 0.09].

Procedure.

Participants had to perform scripted tasks in the VMT-2 as described previously, environmental named “complex”. They must also in a “simple” environment make coffee with milk and two sugars. Therefore, we used the Non-Immersive Coffee Task from [27]. In this VE, participants were projected in a kitchen in which they did not have to move themselves. To achieve the task, objects must have been manipulated on the table - with computer mouse - in a logical order. The two tasks are carried out consecutively in a counterbalanced order. Following the completion of each task, participants were asked to complete two questionnaires. At first, the NASA-TLX was administered. The latter has the advantage to easily allow comparison of tasks while assessing the fluctuation of mental load according to the changing environment. The second questionnaire was AttrakDiff 2 [34 ]. The latter is a subjective measure of the ease of using technology by characterizing the experience in contact with VR. Three subscale are included: (a) the pragmatic qualities of the VR (usability utility); (b) hedonic qualities characterized by pleasure and satisfaction in contact with the VR; (c) the attractiveness or overall quality of the interaction with the VR.

Results.

The results showed a significant age-related difference for the cognitive load requested by the VE. The VMT-2 generated a greater cognitive load in the elderly compared to the young adults [F(1,12) = 8.30; p = 0.017]. Cognitive load generated by the Non-Immersive Coffee Task was similar in both groups [F (1,12) = 3.40; p = 0.617]. A subsequent analysis of the subscales revealed that the main age-related difference in the VMT-2 was in the load time, effort and performance. As for the Non-Immersive Virtual Coffee Task, they differed on the physical demands requested. Moreover, no significant difference between groups and between environments on the UX was found. That was good news because the pragmatic & hedonic aspects of our environment don’t raise the cognitive workload. Indeed, all participants described their experience as pleasant thus satisfying the two VE to which they were exposed. Finally, despite the small sample, some preliminary results have shown that the elderly spend more time in the VMT, they were less efficient in terms of movements, they are more disoriented and they need more time in the training phase compared to the younger adults. In summary, in this pilot study, both environments were considered equal in terms of UX and means of interaction were similar and simplified. The study has demonstrated that the accumulation of tasks to perform in a VE and their associated complexities bring a greater challenge for older adults in terms of requested cognitive resources.

3 Discussion

This paper wants to show how important is the impact of technology on cognition in the elderly. In the beginning, we suppose that VR is a good and reliable/valid means to assess cognitive functions. In the same way, we think that PM is a cognitive function particularly useful to detect pathological decline in aging. But, when we claim that VR is ecologically valid, it is possible to be wrong or to have difficulties to show it, if the cognitive load of the HCI is not considered. To begin this reflection, we conducted 3 very small pilot study with different experimental designs.

After the first pilot study, it was possible to see that the level of stress increased during the immersion in most participants due to navigational and manipulation difficulties in the VE. The limits of this pre-experimental study, in addition to the small number of participants, are seen in the type of measurement selected to observe the cognitive overload (i.e., time to complete the task, blood pressure and heart rate). Other measures would have allowed us to better reflect on cognitive demands of the experimental protocol; thus explaining - at least partially - the fact that the participants did not properly complete the task.

At the end of the second pilot study, we observed, when using a smaller sample that was comparable on multiple variables (i.e. cybersickness, sense of presence, anxiety, executive functions), that: while simplifying the HCI, the more a person must complete tasks in each space, the higher the cognitive load could possibly raise on several dimensions. Finally, the results showed an age effect in the HCI and in the achievement of the VMT-2 tasks.

Concerning the third pilot study, results have shown that the elderly spend more time in the VMT, they were less efficient in terms of movements, they are more disoriented and they need more time in the training phase compared to the younger adults. In summary, in this pilot study, both environments were considered equal in terms of UX and means of interaction were similar and simplified. The study has demonstrated that the accumulation of tasks to perform in a VE and their associated complexities bring a greater challenge for older adults in terms of requested cognitive resources.

To conclude, despite small sample sizes, these 3 pilot studies demonstrated that similar factors increased the solicitation of cognitive resources during task performance. These factors included the means of interaction, the nature of the required tasks and the characteristics of the participants and the VE. These factors were of importance when reinventing the neuropsychological assessment, to make it more ecological by using VR. Hence, the first study suggested that inefficient and complex interfaces could interfere significantly with the neuropsychological assessment thus affecting the validity of the measure. The second study illustrated that not all tasks were equivalent in VE. Indeed, those that required more interaction with the control devices were those that are effortful and were consequently the most frustrating. Lastly, the third study suggested that a larger virtual apartment, in which they are more movements, also influenced the cognitive load - probably through the solicitation of navigation, orientation and memory processes. These components must always be integrated with the neuropsychological measures taken in the VE. Finally, elderly adults appeared to be less skilled with technology. They were consequently more at risk of performing less efficiently in VR than in daily life actions since the measure is taken by a computer. This impacts negatively the sensitivity of the tool for the detection of mild cognitive impairment or for the early identification of neurocognitive disorders such as dementia. Although these age-related changes may fade with the aging of the current young population since they are more familiar with computers and gaming interfaces, it is still essential to consider the current age effects when evaluating in the VR a normal aging population or a clinic population.

References

  1. 1.
    Ellis, J.: Prospective memory or the realization of delayed intentions: a conceptual framework for research. In: Brandimonte, M., Einstein, G.O., McDaniels, M.A. (eds.) Prospective Memory: Theory and Applications, pp. 1–22. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  2. 2.
    Shum, D., Fleming, J.M., Neulinger, K.: Prospective memory and traumatic brain injury: a review. Brain Impairment 3(1), 1–16 (2002)CrossRefGoogle Scholar
  3. 3.
    Kvavilashvili, L., Ellis, J.: Varieties of intention: some distinctions and classifications. In: Brandimonte, M., Einstein, G.O., McDaniel, M.A. (eds.) Prospective Memory: Theory and Applications, pp. 23–51. Lawrence Erlbaum Associates Publishers, Mahwah (1996)Google Scholar
  4. 4.
    Azzopardi, B., Auffray, C., Juhel, J.: L’effet paradoxal du vieillissement sur la mémoire prospective: hypothèses explicatives. Gériatrie et Psychologie Neuropsychiatrie du Vieillissement 13(1), 64–72 (2015)Google Scholar
  5. 5.
    McDaniel, M.A., Einstein, O.G.: Strategic and automatic processes in prospective memory retrieval: a multiprocess framework. Appl. Cogn. Psychol. 14, S127–S144 (2000)CrossRefGoogle Scholar
  6. 6.
    Norman, D.A., Shallice, T.: Attention to action: willed and automatic control of behaviour. In: Davidson, R.J., Schwartz, G.E., Shapiro, D. (eds.) Consciousness and Self-regulation: Advances in Research and Theory. Plenum Press, New York (1986)Google Scholar
  7. 7.
    Burgess, P.W., Shallice, T.: The relationship between prospective and retrospective memory: Neuropsychological evidence. In: Conway, M.A. (ed.) Cognitive Models of Memory, pp. 247–272. The MIT Press, Cambridge (1997)Google Scholar
  8. 8.
    Thompson, C., Henry, J.D., Rendell, P.G., Withall, A., Brodaty, H.: Prospective memory function in mild cognitive impairment and early dementia. J. Int. Neuropsychol. Soc. 16(2), 318–325 (2010)CrossRefGoogle Scholar
  9. 9.
    Van den Berg, E., Kant, N., Postma, A.: Remember to buy milk on the way home! A meta-analytic review of prospective memory in mild cognitive impairment and dementia. J. Int. Neuropsychol. Soc. 18(4), 706–716 (2012)CrossRefGoogle Scholar
  10. 10.
    Lewis, M.W., Babbage, D.R., Leathem, J.M.: Assessing executive performance during cognitive rehabilitation. Neuropsychol. Rehabil. 21(2), 145–163 (2011)CrossRefGoogle Scholar
  11. 11.
    Sbordone, R.J.: Ecological validity: some critical issues for neuropsychologist. In: Sbordone, R.J., Long, C.J. (eds.) Ecological Validity of Neuropsychological Testing, pp. 15–42. St-Lucie Press, New York (1996)Google Scholar
  12. 12.
    Franzen, M.D., Wilhelm, K.L.: Conceptual foundations of ecological validity in neuropsychology. In: Sbordone, D.R.J., Long, C.J. (eds.) Ecological Validity of Neuropsychological Testing, pp. 91–112. St. Lucie Press, Delray Beach (1996)Google Scholar
  13. 13.
    Chaytor, N., Schmitter-Edgecombe, M.: The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills. Neuropsychol. Rev. 13, 181–197 (2003)CrossRefGoogle Scholar
  14. 14.
    Shallice, T., Burgess, P.W.: Deficits in strategy application following frontal lobe damage in man. Brain 114, 727–741 (1991)CrossRefGoogle Scholar
  15. 15.
    Cuberos-Urban, G., Caracuel, A., Vilar-López, Ré, Valls-Serran, C., Bateman, A., Verdejo-García, A.: Ecological validity of the Multiple Errands Test using predictive models of dysexecutive problems in everyday life. J. Clin. Exp. Neuropsychol. 35(3), 329–336 (2013)CrossRefGoogle Scholar
  16. 16.
    Dawson, D.R., Anderson, N.D., Burgess, P., Cooper, E., Krpan, K.M., Stuss, D.: Further development of the multiple errands test: standardized scoring, reliability, and ecological validity for the baycrest version. Arch. Phys. Med. Rehabil. 90(1), S41–S51 (2009)CrossRefGoogle Scholar
  17. 17.
    Knight, C., Alderman, N., Burgess, P.W.: Development of a simplified version of the multiple errands test for use in hospital settings. Neuropsychol. Rehabil. 12(3), 231–255 (2002)CrossRefGoogle Scholar
  18. 18.
    Morris, R.G., Kotitsa, M., Bramham, J., Brooks, B., Rose, F.D.: Virtual reality investigation of strategy formation, rule breaking and prospective memory in patients with focal prefrontal neurosurgical lesions. In: Proceedings of 4th International Conference on Disability, Virtual Reality and Associated Technologies, Hungary, pp. 101–108 (2002)Google Scholar
  19. 19.
    Burgess, P.W., Gonen-Yaacovi, G., Volle, E.: Functional neuroimaging studies of prospective memory: what have we learnt so far? Neuropsychologia 49(8), 2246–2257 (2011)CrossRefGoogle Scholar
  20. 20.
    Pratt, D.R., Zyda, M., Kelleher, K.: Virtual reality: in the mind of the beholder. IEEE Comput. 28(7), 17–19 (1995)Google Scholar
  21. 21.
    Schutheis, M.T., Rizzo, A.A.: The application of virtual reality technology in rehabilitation. Rehabil. Psychol. 46, 296–311 (2001)CrossRefGoogle Scholar
  22. 22.
    Wilson, P.N., Foreman, N., Stanton, D.: Virtual reality, disability and rehabilitation. Disabil. Rehabil. 19(6), 213–220 (1997)CrossRefGoogle Scholar
  23. 23.
    Tarr, M.J., Warren, W.H.: Virtual reality in behavioral neurosciences and beyond. Nat. Neurosci. 5, 1089–1092 (2002)CrossRefGoogle Scholar
  24. 24.
    Zhang, L., Abreu, B.C., Masel, B., Scheibel, R.S., Christiansen, C.H., Huddleston, N., Ottenbacher, K.J.: Virtual reality in the assessment of selective cognitive function after brain injury. Am. J. Phys. Med. Rehabil. 80(8), 597–604 (2001)CrossRefGoogle Scholar
  25. 25.
    Zalla, T., Plassiart, C., Pillon, B., Grafman, J., Sirigu, A.: Action planning in a virtual context after prefrontal cortex damage. Neuropsychologia 39, 759–770 (2001)CrossRefGoogle Scholar
  26. 26.
    Banville, F., Nolin, P., Lalonde, S., Henry, M., Dery, M.P., Villemure, R.: Multitasking and prospective memory: can virtual reality be useful for diagnosis? Behav. Neurol. 23(4), 209 (2010)CrossRefGoogle Scholar
  27. 27.
    Banville, F., Nolin, P.: Using virtual reality to assess prospective memory and executive functions after traumatic brain injury. J. CyberTherapy Rehabil. 5(1), 45–55 (2012)Google Scholar
  28. 28.
    Allain, P., Etcharry-Bouyx, F., Verny, C.: Executive functions in clinical and preclinical Alzheimer’s disease. Revue Neurologique 169(10), 695–708 (2013)CrossRefGoogle Scholar
  29. 29.
    Besnard, J., Richard, P., Banville, F., Nolin, P., Aubin, G., Le Gall, D., Richard, I., Allain, P.: Virtual reality and neuropsychological assessment: The reliability of a virtual kitchen to assess the daily-life activities in victims of traumatic brain injury. Appl. Neuropsychol. Adult 23(3), 223–235 (2016)CrossRefGoogle Scholar
  30. 30.
    Ang, C.S., Zaphiris, P., Mahmood, S.: A model of cognitive loads in massively multiplayer online role playing games. Interact. Comput. 19, 167–179 (2007)CrossRefGoogle Scholar
  31. 31.
    Nelson, B.C., Erlandson, B.: Managing cognitive load in educational multi-user virtual environments: reflection on design practice. Educ. Tech. Res. Dev. 56, 619–641 (2008)CrossRefGoogle Scholar
  32. 32.
    Cicourel, A.V.: Cognitive overload and communication in two healthcare settings. Commun. Med. 1(1), 35–43 (2004)CrossRefGoogle Scholar
  33. 33.
    Banville, F., Nolin, P., Cloutier, J., Bouchard, S.: Description of the Virtual Multitasking Test (V-MT). In: Conference Présented at the Virtual Rehabilitation Conference: From Vision to Reality, Edmonton (2007)Google Scholar
  34. 34.
    Burgess, P.W.: Strategy application disorder: the role of the frontal lobe in human multitasking research. Psychol. Res. 63, 279–288 (2000)CrossRefGoogle Scholar
  35. 35.
    Hassenzahl, M., Burmester, M., Koller, F.: AttrakDiff: Ein Frage-bogen zur Messung wahrgenommener hedonischer und pragmatischerQualität. In: Ziegler, J., Szwillus, G. (eds.) Mensch & Computer 2003. Interaktion in Bewegung, pp. 187–196. B.G. Teubner, Stuttgart (2003)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Université de MontréalMontréalCanada
  2. 2.Centre de développement et de recherche en imagerie numériqueMataneCanada
  3. 3.Université d’AngersAngersFrance

Personalised recommendations