1 Introduction

In conventional usability testing, a questionnaire survey is commonly used to obtain users’ subjective views. By additionally evaluating quantitative factors such as operation time and a N/E ratio, it is possible to show whether the entire system is good or bad, but not where or how improvements should be made. In addition, a large amount of data is required for proper subjective evaluation, and its reproducibility is generally low [1].

To overcome drawbacks specific to subjective testing, the utility of usability testing based on biological information closely related to human cognition was investigated in this study. To design a practical improvement plan, we focused on cognitive processing involved in the handling of a target system or product to reveal which steps impose a heavy cognitive load. To quantify cognitive loads, electroencephalography (EEG) was performed to measure event-related potentials (ERPs) [2, 3]. As shown in Fig. 1, ERPs are represented by a wave that contains information on the cognitive function of information processing. However, because ERPs are triggered at the time of information acquisition, it cannot be used for accurate measurements in situations where the trigger cannot be specified. In this study, we therefore investigated the utility and validity of eye fixation-related potentials (EFRPs) [4], with eye fixation as the onset of information acquisition, in the measurement of cognitive load.

Fig. 1.
figure 1

ERPs wave

Full size image

To reveal quantitative usability testing indices that enable the extraction of problems from the perspective of information processing, we evaluated EFRPs and ERPs under the same task environment to clarify whether they serve as comparable testing indices.

2 Experimental Methods

In a study analyzing the movement of fingers on a touch panel, we extracted movements involved in thinking and searching, showing that it is possible to test usability based on the movement of users. These thinking and searching movements correspond to the two basic motions Plan and Search in Therblig motion analysis. They are also represented by the ERP P300 component, which reflects the task difficulty. As in the study by Ando, the present study was conducted with 9 healthy male and female undergraduate/graduate students.

Experiments are summarized in Fig. 2. In a shielded room, each participant sat in front of a desk and performed visual cognition and visual search tasks (Fig. 3) in a random order. Each task consisting of 10 questions was repeated 3 times in a row to give a total of 30 questions. In visual cognition tasks, after addition and subtraction, the participant input the answer by selecting a number corresponding to the numerical answer from numbers 1–50 displayed on the monitor. In visual search tasks, a number selected randomly from 1–50 was displayed on the monitor, and the participant answered the question by searching the same number out of numbers arranged randomly on the monitor. The participants rested for 3 min between tasks.

Fig. 2.
figure 2

Experimental setup

Full size image
Fig. 3.
figure 3

Visual cognition and visual search tasks

Full size image

A data recorder was used to concurrently record eye movement monitored by an eye tracking device (EyeLink II; SR Research) mounted on the head and electrocardiogram and EEG monitored using a multi-telemeter system (WEB-5000; Nihon Kohden). EEG was performed in accordance with the International 10–20 system [5] with the electrode placed at Fz (frontal lobe) and Cz and Pz (parietal lobe). In addition, an original questionnaire was developed to subjectively evaluate loss of interest, amount of load, difficulty, and stress (on a scale of 1–7) after each task.

3 Experimental Results and Discussion

No significant difference in scores was observed between the two tasks. In addition, no intergroup difference was observed in electrocardiograms (low/high frequency), although load tended to be higher in visual search tasks.

Using the display of each question or eye fixation as a trigger in ERP or EFRP measurement, respectively, EEG data were recorded from Fz, Cz, and Pz between 100 ms before and 1000 ms after trigger onset and were averaged to obtain representative ERP and EFRP waveforms (Fig. 4). As shown in the figure, the cognitive component N2 and the attention and processing component P300 were clearly present, indicating that EEG was measured properly. Therefore, to identify each component, we performed principal component analysis [6] of spatial and temporal information using 30 ERP or EFRP waveforms (30 = 2 tasks × 3 recording sites × 5 subjects). For analysis, 51 potentials were extracted from the graph at a 20-ms interval. Five principal components covering 93.54% were extracted in ERPs, whereas four principal components covering 94.87% was extracted in EFRPs. We measured the peak value of the principal component that increased the loading in the 300-ms latency which is associated with cognition, and the area between 300–124 ms to 300 + 124 ms (176–424 ms) was defined as a characteristic value and was subjected to analysis of variance. A significant difference was observed between Fz and Cz (p < 0.05) in ERPs and between Fz and Cz (p < 0.05) and between Cz and Pz (p < 0.05) in EFRPs (Fig. 5).

Fig. 4.
figure 4

ERP and EFRP waveforms

Full size image
Fig. 5.
figure 5

Results of principal component analysis

Full size image

Electrocardiographic findings and ERPs showed loading tended to increase in visual cognition tasks compared with visual search tasks (p < 0.05) (Fig. 6). In addition, loading was higher at Cz and Pz in the parietal region than at Fz in the frontal region in both ERPs and EFRPs (Fig. 5). Cz and Pz correspond to the somatosensory area and parietal association area in the parietal lobe. The parietal association area is known to reflect visuospatial load in information processing. Our findings suggest that both ERPs and EFRPs increase as the level of difficulty in visual information processing increases, as the processing happens when the target number is found from among the randomly placed numbers. In other words, the present visual search task can be used to evaluate loads associated with searching. However, we were unable to reveal parameters useful for the evaluation of loads involved in thinking. Because mental calculation was involved in the visual cognition tasks, additional cognitive loads might have been needed to calculate and memorize numbers. Because memory generally plays an important role in thinking, ERP and EFRP measurements were performed in the occipitotemporal region (T5, T6, O1, and O2) which reflects information processing involving memory [7].

Fig. 6.
figure 6

Visual search task and frontal, parietal lobe (*: p < 0.05)

Full size image

Figure 7 shows results of an analysis conducted in the same way as the analysis of Fig. 6. The horizontal axis indicates the brain area. One of the systems is for the visual cognition task, and the other system is for the visual search task. In visual cognition tasks, the amplitude of waves tended to be larger at T6 and O2 as expected, indicating that cognitive load involved in visual cognition tasks can be quantified by recording ERPs through the electrodes placed in the occipitotemporal area.

Fig. 7.
figure 7

Visual cognition task and occipitotemporal area (*: p < 0.05)

Full size image

4 Conclusions

In summary, three major findings in this study are as follows:

  1. (1)

    In visual search tasks, which involve visuospatial information processing, cognitive loads can be quantified by recording ERPs in the parietal region.

  2. (2)

    In visual cognition tasks, which are greatly associated with memory in information processing, cognitive loads can be quantified by recording ERPs in the occipitotemporal region.

  3. (3)

    Similar results between EFRPs and ERPs indicate that EFRPs are a practical index for use in usability testing.