Keywords

1 Introduction

According to the World Alzheimer’s Report (2016), the number of dementia patients will continue to increase in the future. The prevalence of dementia patients is expected to demonstrate a three-fold increase from 47 million in 2015 to about 13 million people in 2050. The increase in dementia patients may lead to a range of social problems; consequently, it is urgent to improve the screening and intervention for older adults at risk of and with dementia. Dementia is a disorder that in cognitive function is markedly deduced in comparison to healthy elderly people. There is currently no clear treatment for dementia; it is impossible to recover from dementia to a healthy state once it develops. However, if early detection can be made at the stage of mild cognitive impairment (MCI), which is the precursor state of dementia, recovery or prevention of dementia may be possible. Therefore, in the present situation where no clear treatment for dementia has been established, early screening for MCI can be optimal method as a prevention of dementia. Regarding the screening for MCI, proposals and research are advanced from various viewpoints. Currently, the mainstream in the screening for MCI is the Mini-Mental State Examination (MMSE) that is 30 points questionnaire that is used extensively in clinical and research settings to measure cognitive impairment, such as date of examination execution date. And the cognitive decline is evaluated based on the score. However, problems of this method include low accuracy and length of test time. Previous studies have shown that the accuracy of MMSE is 45–60% sensitivity and 65–90% specificity [1]. Recently, there has also been reported a screening method for MCI using a biomarker. Uchida et. al. published a method to discriminate by means of regression analysis with a multi-marker, which is a combination of the three serum proteins of complement protein, apolipoprotein, and transthyretin among the sequester proteins contained in the blood [2]. This study has shown that by using this method, it is possible to identify whether the cognitive function is normal or MCI with an accuracy of about 80%. However, the problem of this method is invasive and it is necessary to request medical institutions. Thus, screening methods that can be performed easily and with high accuracy is required. Therefore, we focused on the behavioral function which has attracted attention in recent years. It is recently known that the decline in cognitive function appears in behavioral functions, and it is a method that is drawing attention among neuropsychology [3]. In particular, it is reported that MCI patients have obstacles to behavioral functions in instrumental activities of daily living (IADL), which is a complicated behavior accompanied by cognitive activities, among the activities of daily living (ADL), which is an action that we normally do casually [4, 5]. Then, Naturalistic Action Test, which is a screening method for MCI focused on behavioral dysfunction, was represented [6]. This method is evaluating by using human error (HE) occurrence frequency and type of HE during IADL task. We take the video of the experiment condition in which the subject actually performs the IADL task. By using the video, neuropsychologists specify HE. On the other hands, recently, some studies have shown that information technology introduce to improve the method focusing on behavioral dysfunction such as NAT, especially, by introducing Virtual Reality (VR) technology [7]. The IADL task built in this VR environment (VR-IADL) is represented for rehabilitation with dementia and screening for MCI [8, 9]. This system made identifying HE automatically. Martono et al. suggested the algorithm identifying the tendency of HE occurrence using the norm of acceleration data calculated from the finger position data at the time of VR-IADL. Thus, these previous studies are progressing to create a discriminative model of MCI and healthy elderly focusing on error occurrence. However, as a problem of this method, it is necessary to take two steps of defining an error and creating a discriminative model. Also, previous studies focused on the HE occurrence simply, if the nature of behaviors hidden behind the error becomes clear, clinical application is difficult. Therefore, the purpose of this research is to create a discriminative model of MCI and healthy elderly in VR-IADL environment, focusing on action indicators other than HE occurrence. In particular, since the VR-IADL environment can operate with only one tablet terminal, it will be expected to spread in the future. In this report, we generate some feature of MCI with relation between touch interaction, which is a characteristic index unique to the VR-IADL environment, and the composition of IADL tasks and finger position data during VR-IADL task. And we constructed an identification model of MCI and healthy elderly people.

2 Previous Study

In our previous research, we conclude that screen non-contact time, which is the time when finger doesn’t touch on the screen, is longer for MCI patients at the time of VR-IADL. VR-IADL installed in the tablet device outputs the value of 1 and 0, which represent screen contact. If someone touch the screen with his finger, the time interval is defined as Touch Time (TT), and if someone doesn’t touch the screen, the time interval is defined as Non Touch Time (NTT) as shown Fig. 1.

Fig. 1.
figure 1

Definition of touch time and non touch time

Full size image

As a result of the previous study, there was a significant difference between the MCI patients and the healthy elderly people at the 5% level in the sum of the NTT. From this result, it was suggested that the characterization of MCI focusing on touch interaction may be effective. In the previous study, the value of RNTT obtained by normalizing the sum of the NTT with the task achievement time (the sum of the contact time TT of the screen and the finger and the NTT) shown in the expression (1).

$$ RNTT = \frac{{\sum\nolimits_{n = 1}^{N} {NTT_{n} } }}{{\sum\nolimits_{n = 1}^{N} {TT_{n} } + \sum\nolimits_{m = 1}^{M} {NTT_{m} } }} $$
(1)

Then, our previous study has shown that declining cognitive function may be represented by behavioral dysfunction. In this report, we focused on Touch Interaction including this RNTT and analyzed it. Especially, MCI was characterized from the relationship between touch interaction and IADL structure. And we try to analyze finger position data measured in VR-IADL. Lastly, we create a discriminative model of MCI and healthy elderly.

3 Method

In this experiment, various behavior data were measured to characterize MCI in VR-IADL. Touch Interaction data which can be seen from the output data of VR-IADL, finger position data were measured to find behavioral features.

3.1 Participants

Subjects requested cooperation from 14 men and women aged 60 or older living in the state of Pennsylvania, USA. Subjects were asked to undergo various neuropsychological tests shown in Table 1 and MMSE and other subjects on condition that they lived alone, not depression, MCI and healthy subjects were identified [10, 11].

Table 1. Test
Full size table

3.2 Experimental Setup

In this experiment, a tablet device installed VR-IADL application, a web camera for recording the state during the experiment, and a reflection marker and a leap motion sensor for measuring the finger position data are used as shown Fig. 2. The subject sat in front of the tablet device placed on the desk. Reflective marker was attached to the index finger of the subject. The interaction technic is a touch screen, when you touch an object in the VR-IADL environment with your finger, you can have an object. Also, moving a finger while touching the tablet allows you to move objects in the VR-IADL environment. When you release your finger from the screen, you can release the object in the VR-IADL environment. Before doing the task, I explained the operation practice of the VR-IADL, the purpose of the task and points to be noted until the subject understands. In this experiment, Virtual Kitchen System (VK) was used as VR-IADL environment. VK is a non-immersive type VR-IADL application and has two IADL tasks (i) Toast & Coffee Task, (ii) Lunchbox Task. Screen shots of these tasks are shown in Fig. 3. First, Toast & Coffee Task has two tasks of making a cup of instant coffee with sugar and milk and a piece of toast with jam and butter. In order to achieve the task completely, it is necessary to perform a total of 13 steps of operations in combination of the two tasks. Below is the process of Toast and Coffee Task.

Fig. 2.
figure 2

Experimental setup

Full size image
Fig. 3.
figure 3

screen shots the VR-IADL

Full size image
  1. (1)

    place a slice of bread into the toaster;

  2. (2)

    turn on the switch to toast;

  3. (3)

    remove toast and place on the plate;

  4. (4)

    take butter;

  5. (5)

    spread butter on the toast;

  6. (6)

    open the jelly jar;

  7. (7)

    place jelly on the toast with knife;

  8. (8)

    open the coffee jar;

  9. (9)

    take a spoonful of coffee powder with spoon and into the cup;

  10. (10)

    open the sugar pot jar;

  11. (11)

    place sugar into cup with spoon;

  12. (12)

    pour milk into the cup;

  13. (13)

    stir cup of coffee with spoon.

Next, the Lunchbox Task is a task of packing a sandwich with peanut butter and jam with aluminum foil, a juice filled water bottle, and a sweet wrapped in aluminum foil into a lunch box. In order to achieve these tasks completely, it is necessary to carry out a total of 16 steps. The details are shown below.

  1. (1)

    take bread;

  2. (2)

    open the jelly jar;

  3. (3)

    place jelly on the bread with knife;

  4. (4)

    open the peanut butter jar;

  5. (5)

    place peanut butter on bread with knife;

  6. (6)

    take another piece of bread to close sandwich;

  7. (7)

    take a sheet of foil to wrap the sandwich;

  8. (8)

    pack the wrapped sandwich into the lunchbox;

  9. (9)

    take cookies (one by one);

  10. (10)

    take a sheet of foil then wrap the cookies;

  11. (11)

    pack the wrapped cookies into the lunchbox;

  12. (12)

    pour the juice into the thermos;

  13. (13)

    seal the thermos with lid;

  14. (14)

    seal the thermos with cap;

  15. (15)

    pack thermos into the lunch box;

  16. (16)

    close the lunch box

4 Result

4.1 Characterization of MCI Using Touch Interaction

We analyzed touch interaction data at each subtask. The calculated parameters are the sum of Touch Time (TT), the sum of Non-Touch Time (NTT), the number of times of contact with the screen, and time to completion of each task. First, from the measured screen contact/non-contact data, the sum of NTT and RNTT were calculated using the data of the entire task. A Wilcoxon rank sum test showed there was a significant difference (p < .05) in the sum of NTT and RNTT between the healthy participants and MCI participants for both tasks. The data showed that participants with MCI spent more time not making contact with the screen.

Next, touch interaction data were analyzed at each subtask. First, we built an algorithm to show whether the task was achieved and the separate subtasks that comprised each task (breakfast & lunch). The graphs are shown in Fig. 4. The time during which the participant performed the toast task is shown in blue and the time during which the coffee task was performed is shown in green in Fig. 4(a). For these five subtasks (toast, coffee, sandwich, snack and thermos task), the time to completion, the sum of TT, NTT and the number of touch interactions on screen were calculated were analyzed. A Wilcoxon’s rank sum test, showed that there was a significant difference (p < .05) between MCI and healthy elderly in the time to completion, the sum of TT, and NTT for the toast task. There was a trend for a difference (p = .10) between MCI and healthy elderly in the number of the screen contact times on screen in toast task and the sum of TT, NTT in coffee task, the number of the screen contact times on screen and time to completion. The results show that MCI patients showed meaningful differences in toast tasks and coffee tasks compared with healthy elderly people.

Fig. 4.
figure 4

Graph showing the time at which each subtask was performed and the total time required to achieve each task

Full size image

4.2 Characterization Using Autoregressive Model

Finally, the characterization of MCI was carried out by using the norm data of the velocity calculated from finger position data acquired in the VR-IADL environment. For this time, the autoregressive coefficient (m = 10) which autoregressive model generated as shown in (2).

$$ x(l) = \sum\limits_{i = 1}^{m} {a^{m} \left( i \right)x\left( {l - i} \right)} + \epsilon (l) $$
(2)

An autoregressive model is a type of stochastic process expression that predicts time varying data. The reason for introducing this model is the difference in behaviors between MCI patients and healthy elderly. Previous studies have reported that MCI patients have more HE occurrences than healthy elderly people [4].

As a result, this 10-dimensional autoregressive coefficient was calculated and subjected to principal component analysis. As the result of the principal component analysis, the first principal component whose contribution ratio was 90% or more was taken as the feature quantity. From this feature quantity, we identified MCI with healthy elderly people using a linear support vector machine. As a result, both sensitivity and specificity were 0.83 and AUC was 0.94. From this result, it can be said that in the VR-IADL environment, there is a difference in movement of fingers in healthy subjects and MCI patients. The result indicated that MCI patients are constructed as models that include errors occurrence as predictions when modeling changes in the velocity of fingers when performing tasks, and the autoregressive coefficients to be determined are based on healthy elderly and MCI patients. It is predicted that a difference will appear.

5 Create Discriminative Model

We standardized the 20-dimensional feature quantities of the 10-dimensional feature quantity extracted from the relationship between Touch Interaction and subtask and the 20-dimensional feature quantity extracted from the position data of the finger to obtain the Z score, we constructed a discrimination model of healthy elderly and MCI using support vector machine. This ROC analysis is shown in Fig. 5. From this result, the sensitivity was 83% and the specificity was 83%. It is higher than the sensitivity of 45–60% and the specificity 65–90% of MMSE, which is conventionally used in the mainstream. From this, it can be said that even in the VR-IADL environment, the decline in cognitive function affects everyday function.

Fig. 5.
figure 5

ROC analysis

Full size image

6 Conclusion

In this report, we have characterized MCI from touch interaction data measured for each subtask in the VR-IADL environment. Based on the features, we created a discriminative model of MCI or healthy elderly, and confirmed its usefulness. From the data of touch interaction, it was suggested that the toast task and the coffee task performed this time are effective for extracting the characteristics of MCI. In the characterization by autoregressive model using the finger position data, cognitive function affects everyday function. We created a discriminative model of MCI or healthy elderly people using these behavioral features and confirmed its usefulness. As a future prospect, I would like to quantify subtasks and clarify the causal relationship with behavior.