Keywords

1 Introduction

Having a short break during classes is essential. Mental fatigue of a learner is tends to increase over time during a class. In Watanabe et al. [1], the concentration oflearners can be kept for only about 40 min, so it is important to take a break at appropriate intervals in order to conduct efficient learning.

Also, taking breaks at appropriate intervals or changing topics during classes can reduce the fatigue of learners. A study conducted on the efforts of teachers during a class showed that taking breaks during classes were valuable to the learners [2]. However, when a teacher conducts a short break during a class, the decision to conduct the break is based on the experience of the teacher. The proficiency of the teacher is necessary to decide the appropriate timing for the break.

The conception of our research is that teachers can decide the appropriate timing to take a break by understanding the degree of fatigue of the learners. To evaluate the psychological state, there are several questionnaires, such as Profiles of Mood States (POMS) [3]. However, answering the questionnaires often hinders the progress of the class and learning activity. Another method of evaluating psychological state is measuring neurophysiological signals by attaching devices. However, the devices attached to the learner’s body also influence the mental state of the learner and hinder the learning activity.

Physical motions have been reported to represent a specific mental state, one example is leg movements [4]. In this study, we focused on the learner’s leg movement. It is possible to measure the learner’s leg movement without interfering with the progress of the class or attaching measuring devices to the learner. In this research, we propose a method for estimating mental fatigue by measuring the leg movement of the learner.

2 Related Work

There are several studies on estimating human mental fatigue. In [5] mental fatigue is Event related potential (ERP) P300 induced by the oddball task. However, it takes a certain amount time to implement the oddball task and measure the ERP. For this reason, it is difficult to measure ERP during class.

Other studies worked on the estimating the mental fatigue of drivers and pilots using neurophysiological signals such as electroencephalogram (EEG), electrooculography (EOG), heart rate. A review revealed the accuracy of using neurophysiological signals to estimate mental fatigue in drivers and pilots is close to 90% [6]. However, in each measurement, it is necessary to attach a multiple electrodes to the head of the subjects. In addition, it is stated that the number of electrodes is too much for the subjects to be measured comfortably.

Since attaching measuring instruments inhibits learner’s activity, design of a contactless estimation method is needed.

3 Leg Movement for Mental Fatigue Estimation

In the educational setting, the movement of the upper body of a learner in the sitting position in a class changes significantly depending on activities. For example, the expected behavior of the upper body greatly varies between writing and discussing. In such a case, it is difficult to obtain effective measurements of mental fatigue using contactless measurements of upper body movement, due to the variety of tasks.

With regards to lower body posture, there are substantial differences in leg posture. For example in size and physique of a cahir can hinder a specific posture of a learner. And the effect hinder a specific posture of a learner.

Therefore, it is necessary to use measurements that are not easily influenced by the conditions of chairs, as indicators of mental fatigue in the leg movement. As a result, we focused on the number of transitions of leg posture. The definition of a leg posture transition is when a learner starts moving legs and then stops the legs, forming a specific posture. The merit of introducing the transitions of leg posture is that it applies to subjects independently of the physical condition of the subjects.

4 Experiment

The experiment was conducted for 15 undergraduate and postgraduate students (A,...,O) of a science and engineering university. The subjects tackled a 10-min mental arithmetic task (Calc-n) and answered a questionnaire (VAS-n) using the VAS (Visual Analogue Scale) to measure subjective fatigue level. This experiment was repeated for seven repetitions \((\mathrm{n}=1,\dots , 7)\). The mental arithmetic tasks and the questionnaires are conducted on a desktop computer. We did not inform the subjects in advance of the time required and the number of repetitons of the mental arithmetic task that would be conducted during the experiment.

After finishing the tasks (Calc-7 and VAS-7), the subjects took a 20-min break, and then answered the questionnaire again (VAS-break). After that, they tackled the same mental arithmetic task again (Calc-post) and answered the questionnaire (VAS-post).

Before the start of the experiment, a time period for practice was established, to allow the subjects a period to become familiar with answering the mental arithmetic tasks and the questionnaire. This adaptation phase was conducted until the subjects felt that they were fully accustomed (Adaptation Phease). After the practice, the subjects took a 10-min break to dissipate any mental fatigue that might have accumulated during adaptation. Then the subjects answered the questionnaire (VAS-pre) and started the experiment.

The procedure of the experiment is shown in Fig. 1.

Fig. 1.
figure 1

The procedure of the experiment

Full size image

The subjects were instructed not to take a meal and not to look at the clock. The subjects were also instructed not to touch mobile devices such as smartphones during the break to eliminate the influence on the subjects’ mental state by external information. We restricted eating except drinking water during the break, in order to prevent drowsiness.

In order not to hinder the occurrence of mental strain, those that could infer the current time were excluded from the experimental environment so that subjects could not obtain information on the current time. The subjects were instructed to remove their watch. The mental arithmetic tasks were carried out in an unmanned room except the subject so as not to influence the mental state of the subject by the existence of another person.

The leg movement of the subjects during the experiment was recorded on a video except for during the breaks. The number of transitions of the leg posture was counted by observation of the recorded video. The analysis was conducted according to the following rules.

  • It is assumed that a transition happens when the legs move from a stationary state and stop again within 4 s.

  • If the movement stops for 2 s or longer, it is regarded as a stationary state.

  • If the legs keep moving for more than 4 s, it is regarded as a new transition every 4 s.

The mental arithmetic task used in this experiment was to judge whether the calculation formula displayed on the screen was correct or not. The answer from the subject was collected with the mouse connected to the computer. If the calculation formula was thought to be correct, the subjects clicked the left button of the mouse. If the calculation formula was thought to be wrong, the subjects clicked the right button of the mouse. The screenshot of the mental arithmetic task is shown in Fig. 2.

Fig. 2.
figure 2

The screenshot of the mental arithmetic task

Full size image
Fig. 3.
figure 3

The screenshot of the questionnaire

Full size image

In this experiment, VAS was used as a method for evaluating subjective mental fatigue. A questionnaire using VAS was shown on the computer display consisting of the following two questions.

  • Please answer the following questions about your current fatigue.

    • Question 1: Please move the marker to the appropriate place as a degree of “mental” fatigue.

    • Question 2: Please move the marker to the appropriate place as a degree of “physical” fatigue.

The response was made using a slide bar simulating VAS, between the left end of “no feeling of fatigue” and the right end of “maximum imaginable fatigue”. The answer is acquired in 21 steps from 0 to 20 and stored in a CSV file.

The screenshot of the questionnaire is shown in Fig. 3.

5 Results and Discussion

It is necessary to confirm whether the mental arithmetic task used in this study is mental stress that induces a sufficient mental strain on the subjects. The subjective average of the mental and physical fatigue value of the subjects in the questionnaire is shown in Figs. 4 and 5 respectively.

Fig. 4.
figure 4

Average value of the mental fatigue by questionnaire

Full size image
Fig. 5.
figure 5

Average value of the physical fatigue by questionnaire

Full size image

Both subjective mental and physical fatigue were significantly increased prior to the mental arithmetic task (VAS-pre) and after finishing VAS-7. Moreover, there was a significant decrease in the result after a 20-min break (VAS-break). Therefore, it is clear that the mental arithmetic task was appropriate as a mental stress and a physical stress to cause strain on subjects. Also, the subjects recovered by taking a break.

Figure 6 shows the average number of transitions of the leg posture in each mental arithmetic task. The average number of transitions of leg posture in the Calc-7 was significantly higher (p \(\texttt {<}\) .001) compared to the calc-1. Moreover, the average number of transitions of leg posture in the Calc-post was significantly lower (p \(\texttt {<}\) .05) compared to the Calc-7. Although the average number of transitions decreased from the Calc-6 to the Calc-7, there was no significant difference during this period. From these results, it can be considered that the number of transitions increases due to the strain of the mental arithmetic task and decreases due to the break.

Fig. 6.
figure 6

Average number of transitions of the leg posture

Full size image

Next, Spearman’s rank correlation coefficient was calculated for the number of transitions of leg posture in Calc-n \((\mathrm{n}=1,\dots , 7\) post) and the fatigue of VAS-n \((\mathrm{n}=1,\dots , 7\) post) in each subject. The results are shown in Table 1.

Table 1. Correlation between the number of transitions and the fatigue of each subject
Full size table

There was a strong correlation (r \(\texttt {>}\) .60, p \(\texttt {<}\) .05) with 10 out of 15 subjects in the number of transitions of leg posture and mental fatigue. There was also a strong correlation (r \(\texttt {>}\) .60, p \(\texttt {<}\) .05) with ten subjects in the number of transitions of leg posture and physical fatigue.

Spearman’s rank correlation coefficient was calculated for the average number of transitions of leg posture in the Calc-n \((\mathrm{n}=1,\dots , 7\), post) and both the mental and physical fatigue in VAS-n \((\mathrm{n}=1,\dots , 7\), post) of the subjects. The results are shown in Table 2.

Table 2. The correlation between the average number of transitions and the average of each fatigue
Full size table

6 Measuring Device: ThinkingLeg

We designed a device named “ThinkingLeg” to detect leg movement automatically. The device is intended to be placed under the foot of a learner. The developed device is shown in Fig. 7.

Fig. 7.
figure 7

ThinkingLeg

Full size image

ThinkingLeg is a base-like device made up of a large number of photo reflectors and controlled by Raspberry Pi. The detail of the ThinkingLeg is shown in Fig. 8.

Fig. 8.
figure 8

Design of ThinkingLeg

Full size image

The photo reflectors detect foot through the acrylic board. Foot-ground positions are measured every two seconds. By analyzing the change of foot’s ground positions, the number of leg transitions is counted automatically.

7 Conclusion

In this study, we propose a method to estimate mental fatigue by measuring the leg movement of a learner, andwe also conducted an experiment to evaluate the usefulness of the method. A mental arithmetic task was carried out by test subjects in an effort to cause a mental strain. The number of transitions of leg posture was counted. A questionnaire was administered to measure subjective fatigue.

As a result of the experiment, it was found that there was a strong correlation between the number of transitions of the leg posture and subjective fatigue. Therefore, it was revealed that there is a possibility of estimating the fatigue of learners from their leg movement.

As a next step, the number of transitions of the leg posture will be analyzed by multiple persons to improve the objectivity of the results. Also, the performance of ThinkingLeg will be evaluated, and a system that estimates mental fatigue with ThinkingLeg will be developed.